Add MariaDB und phpMyadmin to the docker compose

This commit is contained in:
2025-08-12 22:09:15 +02:00
parent f93664ef89
commit c8813306af
13 changed files with 1090 additions and 70 deletions
+182
View File
@@ -0,0 +1,182 @@
# Data Transfer: SQLite to MariaDB
This document describes how to transfer data from the SQLite test database to the MariaDB container using the maintenance endpoint.
## Overview
The data transfer functionality allows you to migrate all data from your SQLite test database (`./testdata/prepared_test_data.db`) to the MariaDB container. This is useful when:
- Setting up a new MariaDB database with existing test data
- Migrating from SQLite to MariaDB for production use
- Synchronizing test data across different database systems
## Prerequisites
1. **MariaDB Container Running**: Ensure the MariaDB container is up and healthy:
```bash
docker-compose -f docker-compose.dev.yml up mariadb
```
2. **Backend Server Running**: The backend must be running to access the maintenance endpoint:
```bash
docker-compose -f docker-compose.dev.yml up backend-dev
```
3. **SQLite Source Database**: The file `./testdata/prepared_test_data.db` must exist.
## Usage Methods
### Method 1: Using the Shell Script (Recommended)
```bash
# Basic transfer (preserves existing data)
./transfer_sqlite_to_mariadb.sh
# Transfer with data clearing (removes all existing data first)
./transfer_sqlite_to_mariadb.sh clear
```
### Method 2: Direct API Call
```bash
# Basic transfer
curl -X POST http://localhost:8180/api/maintenance/transfer-sqlite-to-mariadb
# Transfer with data clearing
curl -X POST "http://localhost:8180/api/maintenance/transfer-sqlite-to-mariadb?clear=true"
```
### Method 3: Using a HTTP Client (Postman, Insomnia, etc.)
- **Method**: POST
- **URL**: `http://localhost:8180/api/maintenance/transfer-sqlite-to-mariadb`
- **Query Parameter** (optional): `clear=true`
## Transfer Process
The endpoint performs the following steps:
1. **Validation**: Checks if the SQLite source file exists
2. **Database Connections**: Establishes connections to both SQLite and MariaDB
3. **Schema Migration**: Ensures all table structures exist in MariaDB
4. **Data Clearing** (optional): Removes existing data if `clear=true` parameter is provided
5. **Data Copy**: Transfers data in batches for all tables in the correct order
6. **Statistics**: Returns transfer statistics and confirmation
## Supported Tables
The transfer includes all application tables:
### Core Tables
- Users (`user.User`)
- Character data (`models.Char`)
- Skills, Spells, Equipment
- Character properties and relationships
### Learning System Tables
- Sources, Character Classes, Skill Categories
- Learning costs and improvement data
- Spell schools and level costs
### Game Master Data
- Skills, Weapons, Spells, Equipment
- Containers, Transportation, Beliefs
- Character equipment and weapon assignments
## Response Format
### Success Response (HTTP 200)
```json
{
"message": "Data transfer from SQLite to MariaDB completed successfully",
"source_file": "/app/testdata/prepared_test_data.db",
"target": "mariadb:3306/bamort",
"statistics": {
"users": 5,
"chars": 12,
"skills": 45,
"spells": 78,
"equipment": 234
}
}
```
### Error Response (HTTP 4xx/5xx)
```json
{
"error": "SQLite source file not found: /app/testdata/prepared_test_data.db"
}
```
## Configuration Switch
After successful transfer, you may want to switch your application to use MariaDB:
### Update `.env` file:
```bash
# From SQLite configuration
DATABASE_TYPE=sqlite
DATABASE_URL=./testdata/prepared_test_data.db
# To MariaDB configuration
DATABASE_TYPE=mysql
DATABASE_URL=bamort:bG4)efozrc@tcp(mariadb:3306)/bamort?charset=utf8mb4&parseTime=True&loc=Local
```
### Restart the backend:
```bash
docker-compose -f docker-compose.dev.yml restart backend-dev
```
## Important Notes
### ⚠️ Data Safety
- **Backup First**: Always backup your MariaDB data before running with `clear=true`
- **Test Environment**: Consider testing the transfer in a separate environment first
- **Incremental Transfer**: Without `clear=true`, the transfer uses upsert operations (INSERT ... ON DUPLICATE KEY UPDATE)
### 🔧 Performance
- Data is transferred in batches of 100 records for optimal performance
- Large datasets may take several minutes to transfer
- Monitor the backend logs for detailed progress information
### 🐛 Troubleshooting
#### "SQLite source file not found"
- Ensure `./testdata/prepared_test_data.db` exists
- Run from the backend directory
- Check file permissions
#### "Failed to connect to MariaDB target"
- Verify MariaDB container is running and healthy
- Check network connectivity between containers
- Validate MariaDB credentials
#### "Failed to migrate structures"
- Ensure MariaDB user has sufficient privileges
- Check for database schema conflicts
- Review backend logs for specific migration errors
#### Transfer Fails Partway Through
- Tables with foreign key constraints may fail if referenced data is missing
- Use `clear=true` to ensure clean transfer
- Check for data type incompatibilities between SQLite and MariaDB
## Logging
The transfer process generates detailed logs. Monitor them with:
```bash
# View backend logs
docker-compose -f docker-compose.dev.yml logs -f backend-dev
# View MariaDB logs
docker-compose -f docker-compose.dev.yml logs -f mariadb
```
## Related Endpoints
- `GET /api/maintenance/setupcheck` - Verify system status
- `GET /api/maintenance/mktestdata` - Create test data from live database
- `GET /api/maintenance/reconndb` - Reconnect to database
- `GET /api/maintenance/reloadenv` - Reload environment configuration
+497
View File
@@ -531,3 +531,500 @@ func ReloadENV(c *gin.Context) {
config.LoadConfig()
c.JSON(http.StatusOK, gin.H{"message": "Environment variables reloaded successfully"})
}
// TransferSQLiteToMariaDB transfers data from SQLite test database to MariaDB
func TransferSQLiteToMariaDB(c *gin.Context) {
logger.Info("Starte Datenübertragung von SQLite zu MariaDB...")
// Path to the SQLite source database
sourceFile := preparedTestDB
// Check if source file exists
if _, err := os.Stat(sourceFile); os.IsNotExist(err) {
logger.Error("SQLite-Quelldatei nicht gefunden: %s", sourceFile)
respondWithError(c, http.StatusNotFound, "SQLite source file not found: "+sourceFile)
return
}
logger.Debug("SQLite-Quelldatei gefunden: %s", sourceFile)
// Connect to SQLite source database
logger.Debug("Verbinde mit SQLite-Quelldatenbank...")
sourceDB, err := gorm.Open(sqlite.Open(sourceFile), &gorm.Config{})
if err != nil {
logger.Error("Fehler beim Verbinden mit SQLite-Datenbank: %s", err.Error())
respondWithError(c, http.StatusInternalServerError, "Failed to connect to SQLite source: "+err.Error())
return
}
defer func() {
if sqlDB, err := sourceDB.DB(); err == nil {
logger.Debug("Schließe SQLite-Datenbankverbindung")
sqlDB.Close()
}
}()
logger.Debug("SQLite-Verbindung erfolgreich")
// Connect to MariaDB target using the configured connection string
logger.Debug("Verbinde mit MariaDB-Zieldatenbank...")
// Temporarily override config to ensure MariaDB connection
originalType := config.Cfg.DatabaseType
originalURL := config.Cfg.DatabaseURL
originalEnv := config.Cfg.Environment
// Force MariaDB connection parameters
config.Cfg.DatabaseType = "mysql"
config.Cfg.DatabaseURL = "bamort:bG4)efozrc@tcp(mariadb:3306)/bamort?charset=utf8mb4&parseTime=True&loc=Local"
config.Cfg.Environment = "production" // Ensure we don't get test DB
targetDB := database.ConnectDatabaseOrig() // Use original connection method to avoid test DB
// Restore original config
config.Cfg.DatabaseType = originalType
config.Cfg.DatabaseURL = originalURL
config.Cfg.Environment = originalEnv
if targetDB == nil {
logger.Error("Fehler beim Verbinden mit MariaDB-Zieldatenbank")
respondWithError(c, http.StatusInternalServerError, "Failed to connect to MariaDB target")
return
}
logger.Debug("MariaDB-Verbindung erfolgreich")
// Migrate all structures to MariaDB first
logger.Debug("Migriere Strukturen in MariaDB-Datenbank...")
if err := migrateAllStructures(targetDB); err != nil {
logger.Error("Fehler beim Migrieren der Strukturen in MariaDB: %s", err.Error())
respondWithError(c, http.StatusInternalServerError, "Failed to migrate structures to MariaDB: "+err.Error())
return
}
logger.Debug("Strukturen erfolgreich migriert")
// Clear existing data in MariaDB (optional - be careful!)
clearExisting := c.Query("clear")
if clearExisting == "true" {
logger.Info("Lösche bestehende Daten in MariaDB...")
if err := clearMariaDBData(targetDB); err != nil {
logger.Error("Fehler beim Löschen bestehender Daten: %s", err.Error())
respondWithError(c, http.StatusInternalServerError, "Failed to clear existing data: "+err.Error())
return
}
logger.Debug("Bestehende Daten gelöscht")
}
// Copy data from SQLite to MariaDB
logger.Info("Kopiere Daten von SQLite zu MariaDB...")
if err := copySQLiteToMariaDB(sourceDB, targetDB); err != nil {
logger.Error("Fehler beim Kopieren der Daten von SQLite zu MariaDB: %s", err.Error())
respondWithError(c, http.StatusInternalServerError, "Failed to copy data from SQLite to MariaDB: "+err.Error())
return
}
// Get statistics about the transferred data
stats, err := getTestDataStatistics(targetDB)
if err != nil {
logger.Error("Fehler beim Abrufen der Datenstatistiken: %s", err.Error())
respondWithError(c, http.StatusInternalServerError, "Failed to get data statistics: "+err.Error())
return
}
logger.Info("Datenübertragung von SQLite zu MariaDB erfolgreich abgeschlossen")
c.JSON(http.StatusOK, gin.H{
"message": "Data transfer from SQLite to MariaDB completed successfully",
"source_file": sourceFile,
"target": "mariadb:3306/bamort",
"statistics": stats,
})
}
// copySQLiteToMariaDB copies all data from SQLite to MariaDB
func copySQLiteToMariaDB(sqliteDB, mariaDB *gorm.DB) error {
logger.Debug("Starte Kopiervorgang aller Daten von SQLite zu MariaDB...")
// Same table order as copyMariaDBToSQLite but in reverse direction
tables := []interface{}{
// Basis-Strukturen (keine Abhängigkeiten)
&user.User{},
// Learning Costs System - Basis
&models.Source{},
&models.CharacterClass{},
&models.SkillCategory{},
&models.SkillDifficulty{},
&models.SpellSchool{},
// Learning Costs System - Abhängige Tabellen
&models.ClassCategoryEPCost{},
&models.ClassSpellSchoolEPCost{},
&models.SpellLevelLECost{},
&models.SkillCategoryDifficulty{},
&models.SkillImprovementCost{},
// GSMaster Basis-Daten
&models.Skill{},
&models.WeaponSkill{},
&models.Spell{},
&models.Equipment{},
&models.Weapon{},
&models.Container{},
&models.Transportation{},
&models.Believe{},
// Charaktere (Basis)
&models.Char{},
// Charakter-Eigenschaften (abhängig von Char)
&models.Eigenschaft{},
&models.Lp{},
&models.Ap{},
&models.B{},
&models.Merkmale{},
&models.Erfahrungsschatz{},
&models.Bennies{},
&models.Vermoegen{},
// Charakter-Skills (abhängig von Char und Skills)
&models.SkFertigkeit{},
&models.SkWaffenfertigkeit{},
&models.SkAngeboreneFertigkeit{},
&models.SkZauber{},
// Charakter-Equipment (abhängig von Char und Equipment)
&models.EqAusruestung{},
&models.EqWaffe{},
&models.EqContainer{},
}
logger.Info("Kopiere Daten für %d Tabellen von SQLite zu MariaDB...", len(tables))
for i, model := range tables {
logger.Debug("Kopiere Tabelle %d/%d: %T", i+1, len(tables), model)
if err := copyTableDataReverse(sqliteDB, mariaDB, model); err != nil {
logger.Error("Fehler beim Kopieren der Tabellendaten für %T: %s", model, err.Error())
return fmt.Errorf("failed to copy table data for %T: %w", model, err)
}
}
logger.Info("Alle Tabellendaten erfolgreich von SQLite zu MariaDB kopiert")
return nil
}
// copyTableDataReverse copies all data from source to target database
func copyTableDataReverse(sourceDB, targetDB *gorm.DB, model interface{}) error {
tableName := fmt.Sprintf("%T", model)
logger.Debug("Starte Kopiervorgang für Tabelle: %s", tableName)
// Count records in source
var count int64
err := sourceDB.Model(model).Count(&count).Error
if err != nil {
if isTableNotExistError(err) {
logger.Debug("Tabelle %s existiert nicht in der Quelle, überspringe", tableName)
return nil
}
logger.Error("Fehler beim Zählen der Datensätze für %s: %s", tableName, err.Error())
return err
}
if count == 0 {
logger.Debug("Tabelle %s ist leer, keine Daten zu kopieren", tableName)
return nil
}
logger.Debug("Kopiere %d Datensätze für Tabelle %s", count, tableName)
// Copy data in batches
batchSize := 100
totalBatches := (int(count) + batchSize - 1) / batchSize
for batch := 0; batch < totalBatches; batch++ {
offset := batch * batchSize
logger.Debug("Verarbeite Batch %d/%d für Tabelle %s (Offset: %d)", batch+1, totalBatches, tableName, offset)
// Create slice to hold batch data and read from source
var records interface{}
// Read batch from source
switch model.(type) {
case *user.User:
var batch []user.User
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Source:
var batch []models.Source
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.CharacterClass:
var batch []models.CharacterClass
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkillCategory:
var batch []models.SkillCategory
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkillDifficulty:
var batch []models.SkillDifficulty
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SpellSchool:
var batch []models.SpellSchool
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.ClassCategoryEPCost:
var batch []models.ClassCategoryEPCost
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.ClassSpellSchoolEPCost:
var batch []models.ClassSpellSchoolEPCost
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SpellLevelLECost:
var batch []models.SpellLevelLECost
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkillCategoryDifficulty:
var batch []models.SkillCategoryDifficulty
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkillImprovementCost:
var batch []models.SkillImprovementCost
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Skill:
var batch []models.Skill
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.WeaponSkill:
var batch []models.WeaponSkill
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Spell:
var batch []models.Spell
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Equipment:
var batch []models.Equipment
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Weapon:
var batch []models.Weapon
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Container:
var batch []models.Container
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Transportation:
var batch []models.Transportation
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Believe:
var batch []models.Believe
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Char:
var batch []models.Char
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Eigenschaft:
var batch []models.Eigenschaft
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Lp:
var batch []models.Lp
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Ap:
var batch []models.Ap
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.B:
var batch []models.B
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Merkmale:
var batch []models.Merkmale
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Erfahrungsschatz:
var batch []models.Erfahrungsschatz
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Bennies:
var batch []models.Bennies
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.Vermoegen:
var batch []models.Vermoegen
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkFertigkeit:
var batch []models.SkFertigkeit
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkWaffenfertigkeit:
var batch []models.SkWaffenfertigkeit
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkAngeboreneFertigkeit:
var batch []models.SkAngeboreneFertigkeit
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.SkZauber:
var batch []models.SkZauber
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.EqAusruestung:
var batch []models.EqAusruestung
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.EqWaffe:
var batch []models.EqWaffe
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
case *models.EqContainer:
var batch []models.EqContainer
if err := sourceDB.Limit(batchSize).Offset(offset).Find(&batch).Error; err != nil {
return fmt.Errorf("failed to read batch from source: %w", err)
}
records = batch
default:
return fmt.Errorf("unsupported model type: %T", model)
}
// Insert batch into target database using CreateInBatches for better performance
if err := targetDB.Clauses(clause.OnConflict{UpdateAll: true}).CreateInBatches(records, batchSize).Error; err != nil {
logger.Error("Fehler beim Einfügen des Batches für Tabelle %s: %s", tableName, err.Error())
return fmt.Errorf("failed to insert batch for table %s: %w", tableName, err)
}
logger.Debug("Batch %d/%d für Tabelle %s erfolgreich verarbeitet", batch+1, totalBatches, tableName)
}
logger.Debug("Kopiervorgang für Tabelle %s abgeschlossen", tableName)
return nil
}
// clearMariaDBData clears all data from MariaDB tables (use with caution!)
func clearMariaDBData(db *gorm.DB) error {
logger.Debug("Lösche alle Daten aus MariaDB-Tabellen...")
// Clear tables in reverse order due to foreign key constraints
tables := []interface{}{
&models.EqContainer{},
&models.EqWaffe{},
&models.EqAusruestung{},
&models.SkZauber{},
&models.SkAngeboreneFertigkeit{},
&models.SkWaffenfertigkeit{},
&models.SkFertigkeit{},
&models.Vermoegen{},
&models.Bennies{},
&models.Erfahrungsschatz{},
&models.Merkmale{},
&models.B{},
&models.Ap{},
&models.Lp{},
&models.Eigenschaft{},
&models.Char{},
&models.Believe{},
&models.Transportation{},
&models.Container{},
&models.Weapon{},
&models.Equipment{},
&models.Spell{},
&models.WeaponSkill{},
&models.Skill{},
&models.SkillImprovementCost{},
&models.SkillCategoryDifficulty{},
&models.SpellLevelLECost{},
&models.ClassSpellSchoolEPCost{},
&models.ClassCategoryEPCost{},
&models.SpellSchool{},
&models.SkillDifficulty{},
&models.SkillCategory{},
&models.CharacterClass{},
&models.Source{},
&user.User{},
}
for _, model := range tables {
tableName := fmt.Sprintf("%T", model)
logger.Debug("Lösche Daten aus Tabelle: %s", tableName)
if err := db.Session(&gorm.Session{AllowGlobalUpdate: true}).Delete(model).Error; err != nil {
// Continue with other tables even if one fails
logger.Warn("Warnung beim Löschen der Tabelle %s: %s", tableName, err.Error())
}
}
logger.Debug("Alle Tabellendaten gelöscht")
return nil
}
+1
View File
@@ -10,6 +10,7 @@ func RegisterRoutes(r *gin.RouterGroup) {
charGrp.GET("/mktestdata", MakeTestdataFromLive)
charGrp.GET("/reconndb", ReconnectDataBase) // Datenbank neu verbinden
charGrp.GET("/reloadenv", ReloadENV)
charGrp.POST("/transfer-sqlite-to-mariadb", TransferSQLiteToMariaDB) // Transfer data from SQLite to MariaDB
/*
//nur zur einmaligen Ausführung, um das Lernkosten-System zu initialisieren
charGrp.POST("/initialize-learning-costs", InitializeLearningCosts)
+62
View File
@@ -0,0 +1,62 @@
#!/bin/bash
# Script to transfer data from SQLite test database to MariaDB using the maintenance endpoint
# Usage: ./transfer_sqlite_to_mariadb.sh [clear]
BACKEND_URL="http://localhost:8180"
ENDPOINT="/api/maintenance/transfer-sqlite-to-mariadb"
echo "=== Bamort Data Transfer: SQLite to MariaDB ==="
echo ""
# Check if clear parameter is provided
CLEAR_PARAM=""
if [ "$1" == "clear" ]; then
echo "⚠️ WARNING: This will clear all existing data in MariaDB before transfer!"
read -p "Are you sure you want to continue? (y/N): " confirm
if [ "$confirm" != "y" ] && [ "$confirm" != "Y" ]; then
echo "Transfer cancelled."
exit 0
fi
CLEAR_PARAM="?clear=true"
echo "✅ Proceeding with data clearing..."
else
echo "️ Existing data in MariaDB will be preserved (use 'clear' parameter to clear first)"
fi
echo ""
echo "🔄 Starting data transfer..."
echo "📍 Source: ./testdata/prepared_test_data.db (SQLite)"
echo "📍 Target: mariadb:3306/bamort (MariaDB)"
echo ""
# Make the API request
echo "🚀 Calling transfer endpoint..."
response=$(curl -s -w "\n%{http_code}" -X POST "${BACKEND_URL}${ENDPOINT}${CLEAR_PARAM}")
# Extract HTTP status code (last line)
http_code=$(echo "$response" | tail -n1)
# Extract response body (all lines except last)
response_body=$(echo "$response" | head -n -1)
echo ""
if [ "$http_code" = "200" ]; then
echo "✅ Transfer completed successfully!"
echo ""
echo "📊 Response:"
echo "$response_body" | jq '.' 2>/dev/null || echo "$response_body"
else
echo "❌ Transfer failed with HTTP status: $http_code"
echo ""
echo "📋 Error details:"
echo "$response_body" | jq '.' 2>/dev/null || echo "$response_body"
exit 1
fi
echo ""
echo "🎉 Data transfer complete!"
echo ""
echo "💡 Tips:"
echo " - Check the statistics above to verify the transfer"
echo " - Use the backend logs for detailed information"
echo " - Update your .env file to use MariaDB if needed"
+9
View File
@@ -0,0 +1,9 @@
# Environment variables for Bamort production deployment
# Copy this file to .env and adjust the values as needed
# MariaDB Configuration
MARIADB_ROOT_PASSWORD=your_secure_root_password_here
MARIADB_PASSWORD=your_secure_user_password_here
# Application Configuration
# Add other environment variables as needed
+2 -2
View File
@@ -22,8 +22,8 @@ WORKDIR /app
# Copy the compiled binary from builder stage
COPY --from=builder /app/server /app
# Expose port 8080 (or your backend port)
EXPOSE 8080
# Expose port 8180 (backend port)
EXPOSE 8180
# Run the Go server
CMD ["./server"]
-34
View File
@@ -1,34 +0,0 @@
# Alternative Development Dockerfile ohne Air
FROM golang:1.23-alpine
# Install inotify-tools für File-Watching
RUN apk add --no-cache inotify-tools
WORKDIR /app
# Copy go.mod and go.sum first
COPY go.mod go.sum ./
RUN go mod download
# Expose port
EXPOSE 8080
# Simple file watcher script
COPY <<EOF /usr/local/bin/watch-and-run.sh
#!/bin/sh
echo "Starting Go application with file watcher..."
go run ./cmd/main.go &
PID=$!
while inotifywait -r -e modify,create,delete,move . --exclude '(\.git|tmp|uploads|testdata)'; do
echo "Files changed, restarting..."
kill $PID 2>/dev/null
sleep 1
go run ./cmd/main.go &
PID=$!
done
EOF
RUN chmod +x /usr/local/bin/watch-and-run.sh
CMD ["/usr/local/bin/watch-and-run.sh"]
+77 -3
View File
@@ -29,9 +29,26 @@ docker-compose -f docker-compose.dev.yml down
## Verfügbare Services
### MariaDB (Datenbankserver)
- **Port**: 3306 (localhost:3306)
- **Datenbank**: bamort
- **Benutzer**: bamort
- **Passwort**: bG4)efozrc (Development)
- **Root-Passwort**: root_password_dev
- **Persistent**: ✅ Daten bleiben bei Container-Neustarts erhalten
- **Health Check**: ✅ Backend wartet auf Datenbankbereitschaft
### phpMyAdmin (Datenbank-Management)
- **URL**: http://localhost:8082
- **Server**: mariadb (automatisch konfiguriert)
- **Login**: root / root_password_dev (oder bamort / bG4)efozrc)
- **Features**: ✅ Web-basiertes Datenbank-Management
- **Auto-Login**: ✅ Vorkonfiguriert für MariaDB-Container
### Backend (Go mit Air Live-Reloading)
- **URL**: http://localhost:8080
- **API-Dokumentation**: http://localhost:8080/api (falls verfügbar)
- **URL**: http://localhost:8180
- **API-Dokumentation**: http://localhost:8180/api (falls verfügbar)
- **Datenbankverbindung**: Automatisch konfiguriert zu MariaDB-Container
- **Live-Reloading**: ✅ Änderungen an Go-Dateien lösen automatisch einen Neustart aus
- **Volume**: Backend-Code wird vom lokalen Verzeichnis gemountet
@@ -67,6 +84,12 @@ docker-compose -f docker-compose.dev.yml logs frontend-dev
### In Container einsteigen
```bash
# MariaDB
docker exec -it bamort-mariadb-dev mysql -u bamort -p bamort
# phpMyAdmin (Web-Interface)
# Zugriff über Browser: http://localhost:8082
# Backend
docker exec -it bamort-backend-dev sh
@@ -74,10 +97,52 @@ docker exec -it bamort-backend-dev sh
docker exec -it bamort-frontend-dev sh
```
## Datenbank-Management
### Web-Interface (phpMyAdmin)
- **URL**: http://localhost:8082
- **Login**:
- Root: `root` / `root_password_dev`
- User: `bamort` / `bG4)efozrc`
- **Features**: Vollständige Datenbankverwaltung über Web-Interface
### Kommandozeilen-Zugriff
```bash
# Mit mysql client im Container
docker exec -it bamort-mariadb-dev mysql -u bamort -p bamort
# Oder als root
docker exec -it bamort-mariadb-dev mysql -u root -p
```
### Backup/Restore
```bash
# Backup erstellen
docker exec bamort-mariadb-dev mysqldump -u bamort -pbG4)efozrc bamort > backup.sql
# Backup einspielen
docker exec -i bamort-mariadb-dev mysql -u bamort -pbG4)efozrc bamort < backup.sql
```
### Datenbank zurücksetzen
```bash
# Volumes löschen (alle Daten gehen verloren!)
docker-compose -f docker-compose.dev.yml down -v
docker-compose -f docker-compose.dev.yml up --build
```
## Konfiguration
### MariaDB
- Version: 11.4
- Charset: utf8mb4
- Collation: utf8mb4_unicode_ci
- Initialisierungsskripte: `docker/init-db/`
### Backend
- Environment: `development`
- Datenbanktyp: mysql (MariaDB)
- Datenbankverbindung: Automatisch konfiguriert
- Air-Konfiguration: `backend/.air.toml`
- Ausgeschlossene Dateien: Tests, Uploads, temporäre Dateien
@@ -89,7 +154,16 @@ docker exec -it bamort-frontend-dev sh
## Troubleshooting
### Port bereits in Verwendung
Stelle sicher, dass die Ports 8080 und 5173 nicht von anderen Anwendungen verwendet werden.
Stelle sicher, dass die Ports 3306, 8080, 8082, 8180 und 5173 nicht von anderen Anwendungen verwendet werden.
### Datenbankverbindungsfehler
- Warte nach dem Start 10-15 Sekunden, bis MariaDB vollständig initialisiert ist
- Überprüfe die Logs: `docker-compose -f docker-compose.dev.yml logs mariadb`
### phpMyAdmin nicht erreichbar
- Stelle sicher, dass Port 8082 frei ist
- Warte bis MariaDB vollständig gestartet ist
- Überprüfe die Logs: `docker-compose -f docker-compose.dev.yml logs phpmyadmin`
### Node_modules Probleme
Falls es Probleme mit node_modules gibt:
+98
View File
@@ -0,0 +1,98 @@
# Bamort Services Quick Reference
## Development Environment URLs
| Service | URL | Credentials | Description |
|---------|-----|-------------|-------------|
| **Frontend** | http://localhost:5173 | - | Vue.js Application |
| **Backend API** | http://localhost:8180 | - | Go REST API |
| **phpMyAdmin** | http://localhost:8082 | root/root_password_dev | Database Management |
| **MariaDB** | localhost:3306 | bamort/bG4)efozrc | Direct Database Connection |
## Production Environment URLs
| Service | URL | Credentials | Description |
|---------|-----|-------------|-------------|
| **Frontend** | http://localhost:5173 | - | Vue.js Application |
| **Backend API** | http://localhost:8180 | - | Go REST API |
| **MariaDB** | localhost:3306 | bamort/[ENV_VAR] | Direct Database Connection |
> **Note**: phpMyAdmin is disabled in production by default. Uncomment the service in `docker-compose.yml` if needed (Port 8081).
## Container Names
| Service | Development | Production |
|---------|-------------|------------|
| MariaDB | bamort-mariadb-dev | bamort-mariadb |
| Backend | bamort-backend-dev | backend |
| Frontend | bamort-frontend-dev | frontend |
| phpMyAdmin | bamort-phpmyadmin-dev | bamort-phpmyadmin* |
*\*phpMyAdmin is disabled in production by default*
## Quick Commands
### Start Development Environment
```bash
cd /data/dev/bamort/docker
docker-compose -f docker-compose.dev.yml up -d
```
### Start Production Environment
```bash
cd /data/dev/bamort/docker
docker-compose up -d
```
### View Logs
```bash
# All services
docker-compose -f docker-compose.dev.yml logs -f
# Specific service
docker-compose -f docker-compose.dev.yml logs -f mariadb
docker-compose -f docker-compose.dev.yml logs -f backend-dev
docker-compose -f docker-compose.dev.yml logs -f frontend-dev
docker-compose -f docker-compose.dev.yml logs -f phpmyadmin
```
### Database Access
```bash
# Via phpMyAdmin (Browser)
open http://localhost:8082
# Via Command Line
docker exec -it bamort-mariadb-dev mysql -u bamort -p bamort
```
## Port Summary
### Development
- **3306**: MariaDB
- **5173**: Vue.js Frontend (Vite dev server)
- **8080**: Backend API (accessible to frontend)
- **8081**: phpMyAdmin
- **8180**: Go Backend API (container port)
### Production
- **3306**: MariaDB
- **5173**: Vue.js Frontend
- **8081**: phpMyAdmin (disabled by default)
- **8180**: Go Backend API
## Enabling phpMyAdmin in Production
phpMyAdmin is commented out in production for security reasons. To enable it:
1. **Edit `docker-compose.yml`**: Uncomment the phpMyAdmin service section
2. **Set Environment Variables**: Ensure `MARIADB_ROOT_PASSWORD` is set in your `.env` file
3. **Restart Services**: Run `docker-compose up -d phpmyadmin`
Access at: http://localhost:8081
```bash
# Enable phpMyAdmin in production
sed -i 's/^ # phpmyadmin:/ phpmyadmin:/' docker-compose.yml
sed -i 's/^ # / /' docker-compose.yml
docker-compose up -d phpmyadmin
```
+44 -1
View File
@@ -14,7 +14,11 @@ services:
environment:
- GO_ENV=development
- CGO_ENABLED=1
#- CGO_ENABLED=1
- DATABASE_TYPE=mysql
- DATABASE_URL=bamort:bG4)efozrc@tcp(mariadb:3306)/bamort?charset=utf8mb4&parseTime=True&loc=Local
depends_on:
mariadb:
condition: service_healthy
working_dir: /app
# Restart if Go code changes cause crash
restart: unless-stopped
@@ -36,5 +40,44 @@ services:
- backend-dev
restart: unless-stopped
mariadb:
image: mariadb:11.4
container_name: bamort-mariadb-dev
restart: unless-stopped
ports:
- "3306:3306"
environment:
MARIADB_ROOT_PASSWORD: root_password_dev
MARIADB_DATABASE: bamort
MARIADB_USER: bamort
MARIADB_PASSWORD: bG4)efozrc
MARIADB_CHARSET: utf8mb4
MARIADB_COLLATION: utf8mb4_unicode_ci
volumes:
- ./bamort-db-dev:/var/lib/mysql
- ./init-db:/docker-entrypoint-initdb.d
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
start_period: 10s
timeout: 5s
retries: 3
phpmyadmin:
image: phpmyadmin/phpmyadmin:5.2
container_name: bamort-phpmyadmin-dev
restart: unless-stopped
ports:
- "8081:80"
environment:
PMA_HOST: mariadb
PMA_PORT: 3306
PMA_USER: root
PMA_PASSWORD: root_password_dev
MYSQL_ROOT_PASSWORD: root_password_dev
PMA_ARBITRARY: 1
depends_on:
mariadb:
condition: service_healthy
volumes:
go-mod-cache:
+49 -28
View File
@@ -1,35 +1,40 @@
version: "3.8"
services:
# mariadb:
# image: mariadb:10.7
# container_name: my_mariadb
# environment:
# - MYSQL_ROOT_PASSWORD=1234
# - MYSQL_DATABASE=rollenspiel_db
# ports:
# - "3306:3306"
# volumes:
# - db_data:/var/lib/mysql
# command: ['--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
mariadb:
image: mariadb:11.4
container_name: bamort-mariadb
restart: unless-stopped
environment:
MARIADB_ROOT_PASSWORD: ${MARIADB_ROOT_PASSWORD:-secure_root_password}
MARIADB_DATABASE: bamort
MARIADB_USER: bamort
MARIADB_PASSWORD: ${MARIADB_PASSWORD:-secure_user_password}
MARIADB_CHARSET: utf8mb4
MARIADB_COLLATION: utf8mb4_unicode_ci
ports:
- "3306:3306"
volumes:
- ./bamort-db:/var/lib/mysql
- ./init-db:/docker-entrypoint-initdb.d
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
start_period: 10s
timeout: 5s
retries: 3
backend:
build:
context: ../backend
dockerfile: ../docker/Dockerfile
container_name: backend
# environment:
# - YOUR_ENV=example
# environment:
# DB_HOST: mariadb
# DB_USER: root
# DB_PASS: 1234
# DB_NAME: rollenspiel_db
environment:
- DATABASE_TYPE=mysql
- DATABASE_URL=bamort:${MARIADB_PASSWORD:-secure_user_password}@tcp(mariadb:3306)/bamort?charset=utf8mb4&parseTime=True&loc=Local
ports:
- "8080:8080"
# volumes:
# - ./some-local-folder:/app/some-folder
# You can add more configuration as needed.
#depends_on:
# - mariadb
- "8180:8180"
depends_on:
mariadb:
condition: service_healthy
frontend:
build:
@@ -37,12 +42,28 @@ services:
dockerfile: ../docker/Dockerfile.frontend
container_name: frontend
ports:
- "3000:80"
- "5173:80"
depends_on:
- backend
# phpMyAdmin - Database Management (commented out for production)
# Uncomment the following section if you need database management in production
# phpmyadmin:
# image: phpmyadmin/phpmyadmin:5.2
# container_name: bamort-phpmyadmin
# restart: unless-stopped
# ports:
# - "8081:80"
# environment:
# - API_URL=http://backend:8080
# In your frontend code, you'd reference process.env.API_URL or similar
# if using environment variables at build time.
# PMA_HOST: mariadb
# PMA_PORT: 3306
# PMA_USER: root
# PMA_PASSWORD: ${MARIADB_ROOT_PASSWORD:-secure_root_password}
# MYSQL_ROOT_PASSWORD: ${MARIADB_ROOT_PASSWORD:-secure_root_password}
# PMA_ARBITRARY: 1
# depends_on:
# mariadb:
# condition: service_healthy
volumes:
db_data:
+12
View File
@@ -0,0 +1,12 @@
-- Initialize MariaDB database for Bamort application
-- This script runs automatically when the MariaDB container starts for the first time
-- Ensure UTF-8 charset and collation
ALTER DATABASE bamort CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
-- Grant additional privileges to the bamort user if needed
GRANT ALL PRIVILEGES ON bamort.* TO 'bamort'@'%';
FLUSH PRIVILEGES;
-- Log initialization
SELECT 'MariaDB initialization completed for Bamort application' AS message;
+55
View File
@@ -0,0 +1,55 @@
# MariaDB Database Setup
This directory contains initialization scripts for the MariaDB database container used in the Bamort application development environment.
## Configuration
The MariaDB container is configured with the following settings:
- **Image**: mariadb:11.4
- **Database**: bamort
- **Username**: bamort
- **Password**: bG4)efozrc
- **Root Password**: root_password_dev
- **Port**: 3306 (exposed to host)
- **Charset**: utf8mb4
- **Collation**: utf8mb4_unicode_ci
## Volume Mounts
- `mariadb_data`: Persistent data volume for database files
- `./init-db`: Initialization scripts directory
## Initialization Scripts
Scripts in this directory (`init-db/`) are executed automatically when the MariaDB container starts for the first time:
1. `01-init.sql`: Basic database setup and user privileges
## Health Check
The container includes a health check that verifies:
- Database connectivity
- InnoDB engine initialization
## Usage
The MariaDB container starts automatically when running:
```bash
docker-compose -f docker-compose.dev.yml up
```
The backend service will wait for the database to be healthy before starting, thanks to the `depends_on` configuration with `condition: service_healthy`.
## Connection Details
- **Host**: `mariadb` (within Docker network) or `localhost` (from host)
- **Port**: 3306
- **Database**: bamort
- **Username**: bamort
- **Password**: bG4)efozrc
## Data Persistence
Database data is stored in the `mariadb_data` Docker volume and persists across container restarts.