Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add test for ref_input parameter in fused linear preference #468

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

xingyaoww
Copy link
Contributor

This PR adds a test for the ref_input parameter that was introduced in #467.

Changes

  • Add test_ref_input.py to verify the ref_input parameter works correctly in LigerFusedLinearPreferenceBase
  • Test ensures that:
    • Policy model outputs (chosen_logps, rejected_logps) are identical when using ref_input vs not using it
    • Final loss and aux outputs differ when using ref_input vs using input_chunk for reference model
  • Uses same parametrization as other tests for consistency

Testing

The test verifies that:

  1. When ref_input is not provided, the reference model uses input_chunk
  2. When ref_input is provided, the reference model uses it instead of input_chunk
  3. The policy model outputs remain unchanged regardless of ref_input
  4. The final loss differs when using different inputs for the reference model

Follows up on #467 which added ref_input parameter support.

openhands-agent and others added 3 commits December 11, 2024 21:37
- Add test to verify ref_input parameter works correctly in LigerFusedLinearPreferenceBase
- Test ensures policy outputs are identical but losses differ when using ref_input
- Follows PR linkedin#467 which added ref_input parameter support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants