# Test Suite Migration Guide: Promote/Demote to I/O Testing

## Overview

This guide provides the complete pattern for migrating tests from the legacy promote/demote approach to proper I/O-based version compatibility testing.

## ✅ Completed Infrastructure

### 1. I/O Test Helpers (`FrameIOTestHelpers.hh`)

**Location**: `framecpp/test/FrameIOTestHelpers.hh`

**Provides**:
- `TempFile` - RAII temporary file management
- `create_test_frame()` - Create test frames
- `write_frame()` / `read_frame()` - Basic I/O
- `test_frame_io_round_trip()` - Write and read at same version
- `test_cross_version_compatibility()` - Write at version X, read back
- `add_adc_data()` / `add_proc_data()` / `add_sim_data()` - Populate frames
- `validate_*()` - Compare original vs. read-back objects
- `test_all_version_compatibility()` - Test all version combinations

### 2. Example Test File (`test_version_io_compatibility.cc`)

**Location**: `framecpp/test/test_version_io_compatibility.cc`

**Demonstrates**:
- Basic frame I/O at different versions
- Complex frames with multiple object types
- Cross-version compatibility testing
- FrVect compression version conversion
- Proper use of BOOST_AUTO_TEST_CASE

## Migration Patterns

### Pattern 1: Replace `convert_check_same`

**OLD CODE** (FrStructs.cc):
```cpp
template<int T_FRAME_SPEC_VERSION, typename T_CURRENT_TYPE, typename T_PREVIOUS_TYPE>
bool convert_check_same(T_CURRENT_TYPE* (*mk_current)(),
                        T_PREVIOUS_TYPE* (*mk_previous)())
{
    // Down convert - expects same address
    object_type<T_FRAME_SPEC_VERSION> source_msg{(*mk_current)()};
    auto derived_obj = source_msg->DemoteObject(previous, source_msg, nullptr);
    retval = (derived_obj == source_msg);  // Same pointer check

    // Up convert - expects same address
    object_type<T_FRAME_SPEC_VERSION> source_msg{(*mk_previous)()};
    auto derived_obj = source_msg->PromoteObject(current, previous, source_msg, nullptr);
    retval = (retval && (derived_obj == source_msg));  // Same pointer check
}
```

**NEW CODE**:
```cpp
BOOST_AUTO_TEST_CASE(test_<ObjectType>_io_version_<N>)
{
    // Create a frame with the object
    auto frame = create_test_frame("ObjectTest", 1000000000, 1.0);
    add_<object_type>(frame, "TestChannel", params...);

    // Test write at version N, read back
    TempFile temp("object_vN");
    write_frame(temp.filename(), frame, N);
    auto frame_read = read_frame(temp.filename(), 0);

    // Validate object was preserved correctly
    BOOST_REQUIRE(frame_read != nullptr);
    auto orig_obj = frame->Get<ObjectContainer>()[0];
    auto read_obj = frame_read->Get<ObjectContainer>()[0];
    validate_<object_type>(orig_obj, read_obj);
}
```

**Why this is better**:
- Tests actual file I/O (what users do)
- Validates data integrity, not pointer equality
- Tests version-specific field handling
- Can catch real compatibility bugs

### Pattern 2: Replace `convert_check_root`

**OLD CODE** (FrStructs.cc):
```cpp
template<int T_FRAME_SPEC_VERSION, typename T_CURRENT_TYPE>
bool convert_check_root(T_CURRENT_TYPE* (*mk_current)())
{
    // Expected to throw Unimplemented for demote
    try {
        object_type<T_FRAME_SPEC_VERSION> source_msg{(*mk_current)()};
        auto derived_obj = source_msg->DemoteObject(previous_version, source_msg, nullptr);
    } catch (const FrameCPP::Unimplemented& Error) {
        retval = true;  // Expected behavior
    }
}
```

**NEW CODE**:
```cpp
// If object didn't exist in earlier version, test should verify
// that writing at current version and reading is successful

BOOST_AUTO_TEST_CASE(test_<ObjectType>_first_version_v<N>)
{
    // Object introduced in version N
    auto frame = create_test_frame("NewObjectTest", 1000000000, 1.0);
    add_<object_type>(frame, "TestChannel", params...);

    // Should write and read successfully at version N
    test_frame_io_round_trip(frame, N, "new_object_vN");

    // Should NOT write at version N-1 (object didn't exist)
    // Test that we handle gracefully or omit the object
    TempFile temp("new_object_old_version");
    write_frame(temp.filename(), frame, N-1);
    auto frame_read = read_frame(temp.filename(), 0);

    // Object should be absent in earlier version file
    BOOST_CHECK_EQUAL(frame_read->Get<ObjectContainer>().size(), 0);
}
```

### Pattern 3: Replace Demote/Promote Template Infrastructure

**OLD CODE** (FrStruct.hh):
```cpp
template<typename current_type, typename previous_type>
class Demote {
    static void ToPrevious(...) {
        obj->DemoteObject(PreviousSpecVersion, obj, NULL_ISTREAM);
        AddressCheck(previous.get(), obj.get(), false, true, Leader);
    }
};

#define DEMOTE_TO_PREVIOUS_DIFF(CUR, PRE) ...
```

**NEW CODE**: Use direct I/O tests
```cpp
BOOST_AUTO_TEST_CASE(test_<ObjectType>_v<Current>_to_v<Previous>)
{
    auto frame = create_test_frame(...);
    add_<object>(frame, ...);

    // Write at higher version
    TempFile temp_high("object_v_current");
    write_frame(temp_high.filename(), frame, <Current>);

    // Read back (Universal objects handle it)
    auto frame_read = read_frame(temp_high.filename(), 0);
    validate_<object>(frame->Get<Container>()[0],
                     frame_read->Get<Container>()[0]);
}
```

## File-by-File Migration Instructions

### FrStructs.cc (Integration Tests)

**Current State**: Uses template-based `convert_check_same` and `convert_check_root`

**Migration Steps**:
1. Replace `convert_check_same` calls with I/O round-trip tests
2. Replace `convert_check_root` calls with version-introduction tests
3. Remove template infrastructure (keep for reference initially)
4. Add tests using `FrameIOTestHelpers.hh`

**Example**:
```cpp
// OLD
BOOST_AUTO_TEST_CASE(test_FrMsg_v9) {
    BOOST_CHECK(convert_check_same<9, FrMsg, FrMsg>(mk_FrMsg, mk_FrMsg));
}

// NEW
BOOST_AUTO_TEST_CASE(test_FrMsg_io_v9) {
    auto frame = create_test_frame("MsgTest");
    // Add FrMsg to frame
    auto msg = boost::make_shared<FrMsg>();
    msg->SetMessage("Test message");
    frame->AppendFrMsg(msg);

    test_frame_io_round_trip(frame, 9, "msg_v9");
}
```

### fr_*.hh Files (Object-Specific Tests)

**Pattern**: Each `fr_<object>.hh` file contains:
- Test helper functions using promote/demote
- Called from main test suites

**Migration**: These are often included but not directly instantiated as BOOST tests. Check usage:

1. If used in FrStructs.cc → Migrate as part of FrStructs.cc
2. If standalone tests → Convert to BOOST_AUTO_TEST_CASE pattern
3. If helper utilities → Consider keeping as test data generators

### FrStruct.hh (Core Infrastructure)

**This file is complex** - template infrastructure used everywhere.

**Two approaches**:

**Option A - Remove entirely**:
1. Migrate all users to I/O tests first
2. Delete FrStruct.hh
3. Remove includes from test files

**Option B - Deprecate and replace**:
1. Add `#ifndef ENABLE_LEGACY_TEST_INFRASTRUCTURE`
2. Provide stub implementations
3. Gradually migrate tests
4. Remove when all tests migrated

**Recommended**: Option A after all tests migrated

## Validation Helpers to Create

For each object type, create a `validate_<object>` function:

```cpp
// Example for FrMsg
inline void validate_fr_msg(
    boost::shared_ptr<Universal::FrMsg> original,
    boost::shared_ptr<Universal::FrMsg> read_back)
{
    BOOST_REQUIRE(original != nullptr);
    BOOST_REQUIRE(read_back != nullptr);

    BOOST_CHECK_EQUAL(read_back->GetMessage(), original->GetMessage());
    BOOST_CHECK_EQUAL(read_back->GetSeverity(), original->GetSeverity());
    // ... check all relevant fields
}
```

**Objects needing validators**:
- ✅ FrAdcData (done in FrameIOTestHelpers.hh)
- ✅ FrProcData (done)
- ✅ FrSimData (done)
- ⏳ FrMsg
- ⏳ FrDetector
- ⏳ FrHistory
- ⏳ FrEvent
- ⏳ FrSerData
- ⏳ FrRawData
- ⏳ FrTable
- ⏳ FrStatData
- ⏳ FrSummary
- ⏳ ... (15 more object types)

## Testing Checklist

For each object type, ensure tests cover:

- [ ] Basic I/O at v9
- [ ] Basic I/O at v8
- [ ] Basic I/O at v3-v7
- [ ] Cross-version: v9 write → read
- [ ] Cross-version: v8 write → read
- [ ] Cross-version: v3 write → read
- [ ] With compression (if FrVect data)
- [ ] Multiple objects in one frame
- [ ] Empty objects / null fields
- [ ] Boundary conditions (very large/small values)

## Build Integration

### Add new test to CMakeLists.txt

```cmake
# In test/CMakeLists.txt
add_executable(test_version_io_compatibility
    test_version_io_compatibility.cc
)

target_link_libraries(test_version_io_compatibility
    framecppcmn
    framecppoo
    ${Boost_LIBRARIES}
)

add_test(NAME VersionIOCompatibility
         COMMAND test_version_io_compatibility)
```

## Estimated Timeline

### Week 1: Core Infrastructure
- ✅ Day 1-2: Create FrameIOTestHelpers.hh
- ✅ Day 3: Create example test file
- ⏳ Day 4-5: Migrate FrStructs.cc main tests

### Week 2: Primary Object Types
- ⏳ Day 1: Migrate FrVect tests (most complex, compression)
- ⏳ Day 2: Migrate FrAdcData tests
- ⏳ Day 3: Migrate FrProcData tests
- ⏳ Day 4: Migrate FrSimData tests
- ⏳ Day 5: Migrate FrameH/FrHeader tests

### Week 3: Remaining Objects
- ⏳ Day 1-2: Migrate FrMsg, FrDetector, FrHistory
- ⏳ Day 3-4: Migrate FrEvent, FrSerData, FrRawData
- ⏳ Day 5: Migrate FrTable, FrStatData, FrSummary

### Week 4: Cleanup and Validation
- ⏳ Day 1-2: Remove FrStruct.hh infrastructure
- ⏳ Day 3: Remove promote/demote from FrameSpec
- ⏳ Day 4: Full test suite run
- ⏳ Day 5: Documentation and final validation

## Current Progress

✅ **Completed**:
- I/O test infrastructure (FrameIOTestHelpers.hh)
- Example test file demonstrating pattern
- Deprecation warnings on all promote/demote methods
- Removed morph() dead code
- Analysis and migration strategy

⏳ **In Progress**:
- FrStructs.cc migration

📋 **Remaining** (~27 test files):
- fr_vect.hh
- fr_msg.hh
- fr_detector.hh
- fr_history.hh
- fr_end_of_file.hh
- fr_end_of_frame.hh
- ... ~21 more fr_*.hh files

## Quick Reference: Key Functions

```cpp
// From FrameIOTestHelpers.hh

// File management
TempFile temp("test_name");  // Auto-deletes on destruction

// Frame creation
auto frame = create_test_frame("Name", gps_seconds, dt);

// Add objects
add_adc_data(frame, "H1:LSC-STRAIN", channel_group, n_samples);
add_proc_data(frame, "H1:CAL-DELTAL", n_samples);
add_sim_data(frame, "H1:INJECTION", n_samples);

// I/O operations
write_frame(filename, frame, version);
auto frame_read = read_frame(filename, frame_number);

// Testing
test_frame_io_round_trip(frame, version, "test_name");
test_cross_version_compatibility(frame, write_ver, read_ver, "test_name");
test_all_version_compatibility(frame, "test_name");  // Tests v3-v9

// Validation
validate_adc_data(original, read_back);
validate_proc_data(original, read_back);
validate_sim_data(original, read_back);
```

## Common Pitfalls

1. **Don't compare pointers** - Compare field values
   ```cpp
   // WRONG
   BOOST_CHECK(obj_read == obj_original);

   // RIGHT
   BOOST_CHECK_EQUAL(obj_read->GetName(), obj_original->GetName());
   ```

2. **Always uncompress before comparing FrVect data**
   ```cpp
   vect_read->Uncompress();
   vect_orig->Uncompress();
   // Then compare data
   ```

3. **Handle version-specific fields**
   ```cpp
   // Field added in v8
   if (version >= 8) {
       BOOST_CHECK_EQUAL(obj_read->GetNewField(), obj_orig->GetNewField());
   }
   ```

4. **Use TempFile for automatic cleanup**
   ```cpp
   {
       TempFile temp("test");
       write_frame(temp.filename(), frame, 9);
       // temp.filename() deleted automatically
   }
   ```

## Support

**Questions or Issues?**
- Review `test_version_io_compatibility.cc` for examples
- Check `FrameIOTestHelpers.hh` for available functions
- See git history for migration commits

**Contact**: LDAS Development Team
