Prompt Architecture Migration Tracker

Overview

This document tracks the migration of ALL prompt categories from legacy architecture to the new modular GenericPromptBuilder architecture. The bags category serves as the reference implementation.

Current State (December 2024)

✅ bags: Fully migrated to new architecture with modular structure (reference implementation)
✅ 17 categories migrated: hats, hoodies, laptop_skins, letterhead, mobile_apps, mouse_pads, notebook, pen, phone_cases, planner, product_packaging, t-shirts, table_mats, throw_pillows, tumblers, wall_art, water_bottles
❌ 11 additional categories discovered: Still need migration to GenericPromptBuilder architecture
✅ Universal Bulk Generator: Created and working (replaces all category-specific bulk files)
✅ Aspect Ratio Routing: Fixed for proper 9:16 dimensions

🔍 S3 Schema Discovery (December 3, 2024)

IMPORTANT DISCOVERY: S3 schema contains 29 total categories, but only 18 were in our local cache!

Migration Status Summary

Total S3 Categories: 29
Fully Migrated: 18 (bags + 17 completed in current session)
Remaining to Migrate: 11

New Architecture (Bags Reference)

server/config/prompts/bags/
├── main.py              # Main prompt builder using GenericPromptBuilder
├── descriptions.py      # Category-specific descriptions and mappings
└── (no bulk.py needed)  # Universal bulk generator handles all categories

Legacy Architecture (All Other Categories)

server/config/prompts/
├── boxes.py                    # ❌ Single file - needs migration
├── beer_glasses.py            # ❌ Single file - needs migration
├── t-shirts.py                # ❌ Single file - needs migration
└── [25+ other single files]  # ❌ All need migration

Migration Required Categories

✅ COMPLETED MIGRATIONS (18/29)

[x] bags (reference implementation)
[x] hats (migrated)
[x] hoodies (migrated)
[x] laptop_skins (migrated)
[x] letterhead (migrated)
[x] mobile_apps (migrated)
[x] mouse_pads (migrated)
[x] notebook (migrated)
[x] pen (migrated)
[x] phone_cases (migrated)
[x] planner (migrated)
[x] product_packaging (migrated)
[x] t-shirts (migrated)
[x] table_mats (migrated)
[x] throw_pillows (migrated)
[x] tumblers (migrated)
[x] wall_art (migrated)
[x] water_bottles (migrated)

❌ REMAINING TO MIGRATE (11/29) - DISCOVERED FROM S3 SCHEMA

These categories exist in S3 but were not in our original local cache:

Product Categories

[ ] beer_glasses [S3 Status: Completed]
[ ] boxes [S3 Status: Completed]
[ ] coasters [S3 Status: Completed]
[ ] coffee_mugs [S3 Status: Completed]

Apparel Categories

[ ] cushions [S3 Status: Incomplete]

Business/Print Categories

[ ] book_cover [S3 Status: Incomplete]
[ ] brochure [S3 Status: Incomplete]
[ ] business_cards [S3 Status: Incomplete]

Tech/Watch Categories

[ ] digital_watch [S3 Status: Incomplete]

Other Categories

[ ] notepad [S3 Status: Incomplete]
[ ] ooh [S3 Status: Incomplete] (Out-of-Home advertising)

Migration Process (Step-by-Step)

Step 1: Create Category Folder Structure

mkdir server/config/prompts/{category}/

Step 2: Extract Descriptions to descriptions.py

Create server/config/prompts/{category}/descriptions.py:

# Extract all hardcoded strings, mappings, and category-specific logic
# Follow bags/descriptions.py as template

CATEGORY_DESCRIPTIONS = {
    "type": {
        "option1": "description",
        "option2": "description"
    },
    # ... other attributes
}

# Category-specific mappings and functions

Step 3: Create main.py with GenericPromptBuilder

Create server/config/prompts/{category}/main.py:

from config.prompts.generic_prompt_builder import GenericPromptBuilder

def build_prompt(attributes):
    """Build prompt using GenericPromptBuilder architecture"""
    builder = GenericPromptBuilder("category_name", attributes)
    return builder.build()

Step 4: Verify Universal Bulk Generator Compatibility

Test that the universal bulk generator can import config.prompts.{category}.main.build_prompt
Verify __RANDOM__ resolution works properly
Ensure aspect ratio routing works correctly

Step 5: Update Category Schema (if needed)

Update metamock-ai/category_metadata_schema.json if any attribute changes are needed.

Step 6: Test Migration

Single image generation
Bulk image generation
__RANDOM__ attribute resolution
Aspect ratio handling (9:16, etc.)
Round-robin provider selection

Step 7: Remove Legacy File

Delete the old single file: server/config/prompts/{category}.py

Technical Requirements

GenericPromptBuilder Integration

Each category must use GenericPromptBuilder for: - ✅ Universal __RANDOM__ resolution - ✅ Model demographics handling - ✅ Background processing - ✅ Color palette management - ✅ Consistent prompt structure

Universal Bulk Generator Compatibility

Categories must be compatible with: - Dynamic import: config.prompts.{category}.main.build_prompt - AttributeResolver for category-specific attributes - ModelMapper for demographics - BackgroundMapper for backgrounds - ColorMapper for color distribution

Aspect Ratio Support

Ensure all categories work with: - Smart routing (exact_dimensions vs aspect_ratio vs default) - Proper dimension passing to AI providers - Round-robin provider selection

Key Technical Details

Critical Files Modified in This Session

server/backend/services/bulk_gen/bulk_image_service.py - Fixed aspect ratio routing
server/config/prompts/universal_bulk_generator.py - Created universal bulk generator
server/backend/services/bulk_gen/bulk_generator_service.py - Updated to use universal generator

Universal Bulk Generator Features

Single file handles ALL categories
Dynamically imports config.prompts.{category}.main.build_prompt
Uses universal mappers (ModelMapper, ColorMapper)
Initializes category-specific mappers (AttributeResolver, BackgroundMapper)
Eliminates 25+ duplicate bulk files

Aspect Ratio Fix

The critical fix was moving aspect_ratio from nested attributes to root level of input_params:

# BEFORE (broken)
input_params = {
    'attributes': {'aspect_ratio': '9:16'}  # BURIED INSIDE
}

# AFTER (fixed)  
input_params = {
    'aspect_ratio': '9:16',                 # AT ROOT FOR ROUTING
    'attributes': {'aspect_ratio': '9:16'}  # ALSO INSIDE FOR PROVIDERS
}

Migration Benefits

Code Quality

Eliminates massive code duplication
Consistent architecture across all categories
Easier maintenance and updates
Better error handling and logging

Functionality

Universal __RANDOM__ resolution
Proper aspect ratio handling
Consistent prompt structure
Better bulk generation performance

Developer Experience

Single architecture to learn
Predictable file structure
Easier to add new categories
Better testing and debugging

Migration Priority Order

Phase 1 (High Usage Products)

boxes (packaging)
t-shirts (apparel)
coffee_mugs (drinkware)
phone_cases (accessories)

Phase 2 (Business Products)

business_cards
letterhead
wall_art
brochure

Phase 3 (Remaining Categories)

All remaining categories in alphabetical order

Testing Checklist (Per Category)

Basic Functionality

[ ] Single image generation works
[ ] Bulk image generation works
[ ] All category attributes resolve correctly
[ ] Model demographics work (if applicable)
[ ] Background selection works

Advanced Features

[ ] __RANDOM__ resolution works for all attributes
[ ] Color palette distribution works
[ ] Aspect ratio routing works (9:16, 16:9, 1:1)
[ ] Round-robin provider selection works
[ ] Error handling works properly

Integration

[ ] Universal bulk generator can import the category
[ ] Category schema is properly loaded
[ ] S3 analytics tracking works
[ ] Database records are created correctly

Notes for Future Sessions

Context

This conversation was about fixing bulk generation aspect ratio issues
Created universal bulk generator to replace 25+ category-specific files
Fixed aspect ratio routing by moving it to root level of input_params
Successfully tested with bags category showing proper 9:16 dimensions

Current Working State

bags category: ✅ Fully working with new architecture
Universal bulk generator: ✅ Created and working
Aspect ratio fix: ✅ Implemented and tested
All other categories: ❌ Need migration to new architecture

Key Files to Reference

server/config/prompts/bags/main.py - Reference implementation
server/config/prompts/bags/descriptions.py - Reference descriptions
server/config/prompts/universal_bulk_generator.py - Universal bulk system
server/backend/services/bulk_gen/bulk_image_service.py - Aspect ratio fix

CRITICAL: S3 Schema Verification Required

⚠️ IMPORTANT: Always verify attribute values against S3 schema before creating descriptions.py

When creating descriptions.py, DO NOT rely on legacy cached values or assume attribute options. Instead:

# Access backend container and check actual S3 schema
docker exec metamock-backend python config/utils/read_s3_schema.py {category_name}

Key Schema Verification Points: 1. Attribute Names: Check exact spelling and capitalization (e.g., "Fill Level" vs "fill_level") 2. Option Values: Verify all available options match S3 exactly (e.g., "Tempered glass" not just "Glass") 3. Background Structure: Some categories have background_human/background_no_human, others just background 4. New Attributes: S3 schema may have added new attributes not in legacy files

Example Issues Found: - beer_glasses: Legacy had "plastic" material, S3 had "Tempered glass" instead - beer_glasses: Attribute called "Fill Level" (with space) in S3, not "fill_level" - beer_glasses: Single "background" attribute, not split human/no-human like bags

Always use docker exec metamock-backend python config/utils/read_s3_schema.py {category} to get current schema before migration.

Technical Debt Eliminated

Removed 25+ duplicate *_bulk.py files (7,669 lines of code deleted)
Eliminated category-specific bulk generators
Fixed aspect ratio routing causing wrong image dimensions
Improved code maintainability significantly

The migration of remaining categories should follow the exact same pattern established with bags.