Fix multi‑turn loss masking, clarify qwen25/qwen3 types, and strengthen mask tests

[Daucloud]

Summary

• Fixed gen_multi_turn_loss_mask_distill_qwen to properly handle multi‑turn masking.
• Renamed loss‑mask type from "qwen" to "qwen25" and set the default to "qwen3".
• Enhanced loss‑mask correctness tests with additional assertions and scenarios.

Changes

• slime/utils/mask_utils.py: corrected multi‑turn masking and adjusted type names.
• slime/utils/arguments.py: updated loss‑mask type options and default value.
• tests/utils/test_mask_utils.py: added more comprehensive correctness checks.

Testing

• Ran python tests/utils/test_mask_utils.py.
  Running results:

$ python tests/utils/test_mask_utils.py
AMEM [INFO] amem_nccl.cpp:x_init:685 groupID:0 pid:2560859 build:Nov 27 2025 09:15:06 NCCL plugin loaded. pause func:Off offload_free_tag:-1 cuMemEnabled:1
/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
==== Single Turn Test Qwen/Qwen3-0.6B (qwen3) ====
text =  ['<|im_start|>system\nSYSTEM MESSAGE FOR TESTING ONLY<|im_end|>\n<|im_start|>user\nUSER CONTENT FOR TESTING ONLY<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
token_ids =  [151644, 8948, 198, 46487, 40758, 4613, 95535, 26687, 151645, 198, 151644, 872, 198, 6448, 35768, 4613, 95535, 26687, 151645, 198, 151644, 77091, 198, 151667, 271, 151668, 271, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151645, 198]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['<think>\n\n</think>\n\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
==== Single Turn Test Qwen/Qwen2.5-0.5B-Instruct (qwen25) ====
text =  ['<|im_start|>system\nSYSTEM MESSAGE FOR TESTING ONLY<|im_end|>\n<|im_start|>user\nUSER CONTENT FOR TESTING ONLY<|im_end|>\n<|im_start|>assistant\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
token_ids =  [151644, 8948, 198, 46487, 40758, 4613, 95535, 26687, 151645, 198, 151644, 872, 198, 6448, 35768, 4613, 95535, 26687, 151645, 198, 151644, 77091, 198, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151645, 198]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['ASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
==== Single Turn Test deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (distill_qwen) ====
text =  ['<｜begin▁of▁sentence｜>SYSTEM MESSAGE FOR TESTING<｜User｜>USER CONTENT FOR TESTING ONLY<｜Assistant｜>ASSISTANT RESPONSE FOR TESTING ONLY<｜end▁of▁sentence｜>']
token_ids =  [151646, 46487, 40758, 4613, 95535, 151644, 6448, 35768, 4613, 95535, 26687, 151645, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151643]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['ASSISTANT RESPONSE FOR TESTING ONLY<｜end▁of▁sentence｜>']
==== Multi-turn with Tools Test Qwen/Qwen3-0.6B (qwen3) ====
text =  ['<|im_start|>system\nSYSTEM MESSAGE FOR TESTING ONLY\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>\n{"type": "function", "function": {"name": "terminal", "description": "Perform operations from the terminal.", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "The bash command to execute as `bash -c <command>`"}, "description": {"type": "string", "description": "Brief description of the command for the user."}}, "required": ["command"]}}}\n{"type": "function", "function": {"name": "read_file", "description": "Read the content of a file given its path.", "parameters": {"type": "object", "properties": {"file_path": {"type": "string", "description": "The absolute path to the file to be read."}}, "required": ["file_path"]}}}\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{"name": <function-name>, "arguments": <args-json-object>}\n</tool_call><|im_end|>\n<|im_start|>user\nUSER CONTENT FOR TESTING ONLY<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nI WILL CALL terminal\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call>\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call><|im_end|>\n<|im_start|>user\n<tool_response>\nLICENSE  README.md  README_zh.md\n</tool_response><|im_end|>\n<|im_start|>user\n<tool_response>\nLICENSE  README.md  README_zh.md\n</tool_response><|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
token_ids =  [151644, 8948, 198, 46487, 40758, 4613, 95535, 26687, 271, 2, 13852, 271, 2610, 1231, 1618, 825, 476, 803, 5746, 311, 7789, 448, 279, 1196, 3239, 382, 2610, 525, 3897, 448, 729, 32628, 2878, 366, 15918, 1472, 15918, 29, 11874, 9492, 510, 27, 15918, 397, 4913, 1313, 788, 330, 1688, 497, 330, 1688, 788, 5212, 606, 788, 330, 36327, 497, 330, 4684, 788, 330, 46951, 7525, 504, 279, 15022, 10465, 330, 13786, 788, 5212, 1313, 788, 330, 1700, 497, 330, 13193, 788, 5212, 5631, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 785, 27023, 3210, 311, 9026, 438, 1565, 46216, 481, 66, 366, 5631, 43626, 14345, 330, 4684, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 85984, 4008, 315, 279, 3210, 369, 279, 1196, 1189, 38154, 330, 6279, 788, 4383, 5631, 1341, 3417, 532, 4913, 1313, 788, 330, 1688, 497, 330, 1688, 788, 5212, 606, 788, 330, 878, 2458, 497, 330, 4684, 788, 330, 4418, 279, 2213, 315, 264, 1034, 2661, 1181, 1815, 10465, 330, 13786, 788, 5212, 1313, 788, 330, 1700, 497, 330, 13193, 788, 5212, 1192, 2638, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 785, 10740, 1815, 311, 279, 1034, 311, 387, 1349, 1189, 38154, 330, 6279, 788, 4383, 1192, 2638, 1341, 3417, 532, 522, 15918, 1339, 2461, 1817, 729, 1618, 11, 470, 264, 2951, 1633, 448, 729, 829, 323, 5977, 2878, 220, 151657, 151658, 11874, 9492, 510, 151657, 198, 4913, 606, 788, 366, 1688, 11494, 8066, 330, 16370, 788, 366, 2116, 56080, 40432, 31296, 151658, 151645, 198, 151644, 872, 198, 6448, 35768, 4613, 95535, 26687, 151645, 198, 151644, 77091, 198, 151667, 271, 151668, 271, 40, 27915, 25752, 15022, 198, 151657, 198, 4913, 606, 788, 330, 36327, 497, 330, 16370, 788, 5212, 5631, 788, 330, 4730, 95642, 151658, 198, 151657, 198, 4913, 606, 788, 330, 36327, 497, 330, 16370, 788, 5212, 5631, 788, 330, 4730, 95642, 151658, 151645, 198, 151644, 872, 198, 151665, 198, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 198, 151666, 151645, 198, 151644, 872, 198, 151665, 198, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 198, 151666, 151645, 198, 151644, 77091, 198, 151667, 271, 151668, 271, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151645, 198]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['<think>\n\n</think>\n\nI WILL CALL terminal\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call>\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call><|im_end|>\n', '<think>\n\n</think>\n\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
==== Multi-turn with Tools Test Qwen/Qwen2.5-0.5B-Instruct (qwen25) ====
text =  ['<|im_start|>system\nSYSTEM MESSAGE FOR TESTING ONLY\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>\n{"type": "function", "function": {"name": "terminal", "description": "Perform operations from the terminal.", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "The bash command to execute as `bash -c <command>`"}, "description": {"type": "string", "description": "Brief description of the command for the user."}}, "required": ["command"]}}}\n{"type": "function", "function": {"name": "read_file", "description": "Read the content of a file given its path.", "parameters": {"type": "object", "properties": {"file_path": {"type": "string", "description": "The absolute path to the file to be read."}}, "required": ["file_path"]}}}\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{"name": <function-name>, "arguments": <args-json-object>}\n</tool_call><|im_end|>\n<|im_start|>user\nUSER CONTENT FOR TESTING ONLY<|im_end|>\n<|im_start|>assistant\nI WILL CALL terminal\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call>\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call><|im_end|>\n<|im_start|>user\n<tool_response>\nLICENSE  README.md  README_zh.md\n</tool_response><|im_end|>\n<|im_start|>user\n<tool_response>\nLICENSE  README.md  README_zh.md\n</tool_response><|im_end|>\n<|im_start|>assistant\nASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
token_ids =  [151644, 8948, 198, 46487, 40758, 4613, 95535, 26687, 271, 2, 13852, 271, 2610, 1231, 1618, 825, 476, 803, 5746, 311, 7789, 448, 279, 1196, 3239, 382, 2610, 525, 3897, 448, 729, 32628, 2878, 366, 15918, 1472, 15918, 29, 11874, 9492, 510, 27, 15918, 397, 4913, 1313, 788, 330, 1688, 497, 330, 1688, 788, 5212, 606, 788, 330, 36327, 497, 330, 4684, 788, 330, 46951, 7525, 504, 279, 15022, 10465, 330, 13786, 788, 5212, 1313, 788, 330, 1700, 497, 330, 13193, 788, 5212, 5631, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 785, 27023, 3210, 311, 9026, 438, 1565, 46216, 481, 66, 366, 5631, 43626, 14345, 330, 4684, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 85984, 4008, 315, 279, 3210, 369, 279, 1196, 1189, 38154, 330, 6279, 788, 4383, 5631, 1341, 3417, 532, 4913, 1313, 788, 330, 1688, 497, 330, 1688, 788, 5212, 606, 788, 330, 878, 2458, 497, 330, 4684, 788, 330, 4418, 279, 2213, 315, 264, 1034, 2661, 1181, 1815, 10465, 330, 13786, 788, 5212, 1313, 788, 330, 1700, 497, 330, 13193, 788, 5212, 1192, 2638, 788, 5212, 1313, 788, 330, 917, 497, 330, 4684, 788, 330, 785, 10740, 1815, 311, 279, 1034, 311, 387, 1349, 1189, 38154, 330, 6279, 788, 4383, 1192, 2638, 1341, 3417, 532, 522, 15918, 1339, 2461, 1817, 729, 1618, 11, 470, 264, 2951, 1633, 448, 729, 829, 323, 5977, 2878, 220, 151657, 151658, 11874, 9492, 510, 151657, 198, 4913, 606, 788, 366, 1688, 11494, 8066, 330, 16370, 788, 366, 2116, 56080, 40432, 31296, 151658, 151645, 198, 151644, 872, 198, 6448, 35768, 4613, 95535, 26687, 151645, 198, 151644, 77091, 198, 40, 27915, 25752, 15022, 198, 151657, 198, 4913, 606, 788, 330, 36327, 497, 330, 16370, 788, 5212, 5631, 788, 330, 4730, 95642, 151658, 198, 151657, 198, 4913, 606, 788, 330, 36327, 497, 330, 16370, 788, 5212, 5631, 788, 330, 4730, 95642, 151658, 151645, 198, 151644, 872, 198, 27, 14172, 9655, 397, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 198, 522, 14172, 9655, 29, 151645, 198, 151644, 872, 198, 27, 14172, 9655, 397, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 198, 522, 14172, 9655, 29, 151645, 198, 151644, 77091, 198, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151645, 198]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['I WILL CALL terminal\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call>\n<tool_call>\n{"name": "terminal", "arguments": {"command": "ls"}}\n</tool_call><|im_end|>\n', 'ASSISTANT RESPONSE FOR TESTING ONLY<|im_end|>\n']
==== Multi-turn with Tools Test deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (distill_qwen) ====
text =  ['<｜begin▁of▁sentence｜>SYSTEM MESSAGE FOR TESTING<｜User｜>USER CONTENT FOR TESTING ONLY<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>terminal\n```json\n{"command": "ls"}\n```<｜tool▁call▁end｜>\n<｜tool▁call▁begin｜>function<｜tool▁sep｜>terminal\n```json\n{"command": "ls"}\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜><｜end▁of▁sentence｜><｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>LICENSE  README.md  README_zh.md<｜tool▁output▁end｜><｜tool▁outputs▁end｜><｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>LICENSE  README.md  README_zh.md<｜tool▁output▁end ｜><｜tool▁outputs▁end｜><｜Assistant｜>ASSISTANT RESPONSE FOR TESTING ONLY<｜end▁of▁sentence｜>']
token_ids =  [151646, 46487, 40758, 4613, 95535, 151644, 6448, 35768, 4613, 95535, 26687, 151645, 27, 130957, 14172, 10417, 223, 44620, 10417, 223, 7265, 130957, 1784, 130957, 14172, 10417, 223, 6659, 10417, 223, 7265, 130957, 29, 1688, 27, 130957, 14172, 10417, 223, 28036, 130957, 29, 36327, 198, 73594, 2236, 198, 4913, 5631, 788, 330, 4730, 16707, 73594, 27, 130957, 14172, 10417, 223, 6659, 10417, 223, 408, 130957, 397, 27, 130957, 14172, 10417, 223, 6659, 10417, 223, 7265, 130957, 29, 1688, 27, 130957, 14172, 10417, 223, 28036, 130957, 29, 36327, 198, 73594, 2236, 198, 4913, 5631, 788, 330, 4730, 16707, 73594, 27, 130957, 14172, 10417, 223, 6659, 10417, 223, 408, 130957, 1784, 130957, 14172, 10417, 223, 44620, 10417, 223, 408, 130957, 29, 151643, 27, 130957, 14172, 10417, 223, 41006, 10417, 223, 7265, 130957, 1784, 130957, 14172, 10417, 223, 3006, 10417, 223, 7265, 130957, 29, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 27, 130957, 14172, 10417, 223, 3006, 10417, 223, 408, 130957, 1784, 130957, 14172, 10417, 223, 41006, 10417, 223, 408, 130957, 29, 27, 130957, 14172, 10417, 223, 41006, 10417, 223, 7265, 130957, 1784, 130957, 14172, 10417, 223, 3006, 10417, 223, 7265, 130957, 29, 64368, 220, 61945, 21324, 220, 61945, 6415, 71, 21324, 27, 130957, 14172, 10417, 223, 3006, 10417, 223, 408, 130957, 1784, 130957, 14172, 10417, 223, 41006, 10417, 223, 408, 130957, 29, 151645, 4939, 3846, 2821, 76173, 4613, 95535, 26687, 151643]
loss_mask =  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1]
selected_texts =  ['<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>terminal\n```json\n{"command": "ls"}\n```<｜tool▁call▁end｜>\n<｜tool▁call▁begin｜>function<｜tool▁sep｜>terminal\n```json\n{"command": "ls"}\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜><｜end▁of▁sentence｜>', 'ASSISTANT RESPONSE FOR TESTING ONLY<｜end▁of▁sentence｜>']
AMEM [INFO] amem_nccl.cpp:x_fini:693 groupID:0 pid:2560859 exit

[Surya-Gunukula]

#1159, also implemented similar loss masking