Predicted tokens support in instructor

[sreeprasannar]

#1145 is trivially supported by instructor (since we just pass on the kwargs to the openai endpoint). This PR bubbles up predicted tokens details in the total usage in update_total_usage from the response which can be useful for callers.

Also added a test to demonstrate the correct behaviour

---

Important

Add tracking of predicted tokens in update_total_usage and test in test_predicted_outputs.py.

• Behavior:
  • update_total_usage in utils.py now tracks accepted_prediction_tokens and rejected_prediction_tokens from completion_tokens_details.
• Testing:
  • New test test_predicted_outputs in test_predicted_outputs.py verifies tracking of predicted tokens in response usage.

^{This description was created by for 3f38122. It will automatically update as commits are pushed.}

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 3f38122 in 21 seconds

More details

• Looked at 53 lines of code in 2 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. tests/llm/test_openai/test_predicted_outputs.py:3

• Draft comment:
  The client parameter is not initialized in this test function. Ensure that client is properly set up before using it in the test.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The test function is missing a proper setup for the client parameter, which is crucial for the test to run correctly. This could lead to test failures or unexpected behavior.

2. tests/llm/test_openai/test_predicted_outputs.py:33

• Draft comment:
  Assertions should include error messages for better debugging. This applies to both assertions in this file.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The assertions in the test file lack error messages, which is against the rule that assertions should always have an error message.

Workflow ID: wflow_01M0obeWPvzXT3GQ

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.