Release 3.0.4 (#206)

* V3 (#187)

* V3

* removing v2 ci pipeline

* removing outdated .gitmodules

* keeping the noise that sample size of 50 and slight decay of .5 in EMA provides to avoid having any one model completely dominate the subnet.

* release 2.2.6 datasets, models, and lora support (#188)

* deprecate stable-diffusion-inpainting

* .env templates

* V3/RGB (#191)

* bgr images --> rgb images

* proper BGR -> RGB conversion

* eradicate all usage of bgr in image challenge flow

* extract frames as rgb

* skip extraneous rgb conversion

* fix deeperforesnics consistency

* v2 frame sampling parity + eidon mp4 fix

* missing import

* handling improper reporting of fps in wembs

* correct content-type on miner side

* max_fpx setting

* improved video metadata extraction

* cleaning up ffprobe options

* fixing first frame rotation edge case

* i2i fix

---------

Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* V3 frame extraction (#192)

* bgr images --> rgb images

* proper BGR -> RGB conversion

* eradicate all usage of bgr in image challenge flow

* extract frames as rgb

* skip extraneous rgb conversion

* fix deeperforesnics consistency

* v2 frame sampling parity + eidon mp4 fix

* missing import

* handling improper reporting of fps in wembs

* correct content-type on miner side

* max_fpx setting

* improved video metadata extraction

* cleaning up ffprobe options

* fixing first frame rotation edge case

* i2i fix

* frame extraction

---------

Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* setup.sh

* removing wandb log call from generator

* V3/2.2.9 (#189)

* mugshot dataset

* black

* i2v support and fixed prompt motion enhancement

* gen pipeline updates for i2v

* fixing prompt indexing

* properly handling new prompt dictionary key (task type)

* V3/2.2.11 (#190)

* mugshot dataset

* black

* i2v support and fixed prompt motion enhancement

* gen pipeline updates for i2v

* prompt sanitation + i2v model

* more retries for prompt sanitation

* fixing truthy tuple assertion

* Update min_compute.yml

* fixing setup script name in docs

* correct script name

* updated requirements.txt with bittensor-cli

* removing wandb.off

* import cleanup

* miner substrate thread restart + vali autoupdate test

* temporary v3 branch set to test autoudpate

* autoupdate update

* lower frequency of audoupdate check

* autoudpate test

* check autoupdate at setp 0

* typo

* autoupdate test

* dont set weights immediately at startup in case of many restarts

* Pyproject toml (#193)

* pyproject setup

* executable setup.sh

* autoupdate test

* resetting version after autoupdate tests

* Add Hugging Face model access instructions to validator docs; improve logging and fix LLM device mapping for multi-GPU

- Added section to Validating.md with instructions for gaining access to required Hugging Face models (FLUX.1-dev, DeepFloyd IF).
- Added logging of generation arguments in generation_pipeline.py.
- Fix LLM loading for multi-GPU in prompt_generator.py: use device_map and remove .to(self.device) for quantized models.
  Quantized LLMs must use device_map for correct device placement; calling .to(self.device) causes device mismatch errors. Parse GPU ID from device string for device_map assignment.

* fixing image_samples check for i2i

* hf_xet requirement

* wandb autorestart

* Fix: raise error if image is None for i2i/i2v tasks and ensure image is converted from array

* fixing wandb autorestart

* error log

* Update setup.sh to install Node.js 20.x LTS from NodeSource for pm2 compatibility; add doc note for existing validators' Hugging Face access

* external port for proxy cuz tensordock rugged us (#196)

* incentive doc

* Typo

* proxy updates

* v2 parity encoding (#197)

* final autoupdate test

* reset version

---------

Co-authored-by: Benjamin S Liang <caliangben@gmail.com>
Co-authored-by: Dylan Uys <dylan@bitmind.ai>

* autoupdate set to main

* testing autoupdate on testnet

* autoupdate enabled by default

* autoudpate testnet

* pointing autoupdate at main by default

* removing extra state load command

* setting back to 360 epoch length

* burn for initial v3 release rampup

* debug log typo

* fixed merge to testnet

* Max Frames and Timeout (#203)

* fixing wandb cache clean paths (#202)

* max frames configuration

* fn header update

* slight increase to timeout

* adding extra metadata to testnet requests for miners (#201)

* remove max size arg

* Testnet Metadata (#204)

* adding extra metadata to testnet requests for miners

* adding label and mediatype to testnet metadata

* Log Augmentation Parameters (#205)

* log augmentation params

* braindead typo

* bump verison

---------

Co-authored-by: Benjamin S Liang <caliangben@gmail.com>
Co-authored-by: Dylan Uys <dylan@bitmind.ai>

Author: dylanuys

VERSION

@@ -1 +1 @@
-3.0.3
+3.0.4

bitmind/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "3.0.3"
+__version__ = "3.0.4"
 
 version_split = __version__.split(".")
 __spec_version__ = (

bitmind/cache/sampler/video_sampler.py

@@ -39,6 +39,7 @@ async def sample(
         remove_from_cache: bool = False,
         min_duration: float = 1.0,
         max_duration: float = 6.0,
+        max_frames: int = 144,
     ) -> Dict[str, Any]:
         """
         Sample random video segments from the cache.
@@ -71,6 +72,7 @@ async def sample(
                 files=cached_files,
                 min_duration=min_duration,
                 max_duration=max_duration,
+                max_frames=max_frames,
                 remove_from_cache=remove_from_cache,
             )
 
@@ -85,6 +87,7 @@ async def _sample_frames(
         min_duration: float = 1.0,
         max_duration: float = 6.0,
         max_fps: float = 30.0,
+        max_frames: int = 144,
         remove_from_cache: bool = False,
         as_float32: bool = False,
         channels_first: bool = False,
@@ -94,8 +97,11 @@ async def _sample_frames(
         Sample a random video segment and return it as a numpy array.
 
         Args:
+            files: Dict mapping source names to lists of video file paths
             min_duration: Minimum duration of video segment to extract in seconds
             max_duration: Maximum duration of video segment to extract in seconds
+            max_fps: Maximum frame rate to use when sampling frames
+            max_frames: Maximum number of frames to extract
             remove_from_cache: Whether to remove the source video from cache
             as_float32: Whether to return frames as float32 (0-1) instead of uint8 (0-255)
             channels_first: Whether to return frames with channels first (TCHW) instead of channels last (THWC)
@@ -155,6 +161,7 @@ async def _sample_frames(
                 target_duration = min(target_duration, total_duration)
 
                 num_frames = int(target_duration * frame_rate) + 1
+                num_frames = min(num_frames, max_frames)
 
                 actual_duration = (num_frames - 1) / frame_rate
 

bitmind/config.py

@@ -157,7 +157,7 @@ def add_validator_args(parser):
         "--neuron.miner-total-timeout",
         type=float,
         help="Total timeout for miner requests in seconds",
-        default=9.0,
+        default=11.0,
     )
 
     parser.add_argument(
@@ -300,6 +300,13 @@ def add_validator_args(parser):
         default=6.0,
     )
 
+    parser.add_argument(
+        "--challenge.max-frames",
+        type=int,
+        help="Maximum number of video frames to sample for a challenge",
+        default=144,
+    )
+
 
 def add_data_generator_args(parser):
     parser.add_argument(

bitmind/epistula.py

@@ -101,6 +101,7 @@ async def query_miner(
     total_timeout: float,
     connect_timeout: Optional[float] = None,
     sock_connect_timeout: Optional[float] = None,
+    testnet_metadata: dict = None,
 ) -> Dict[str, Any]:
     """
     Query a miner with media data.
@@ -130,16 +131,22 @@ async def query_miner(
 
     try:
 
-        headers = generate_header(hotkey, media, axon_info.hotkey)
         url = f"http://{axon_info.ip}:{axon_info.port}/detect_{modality}"
+        headers = generate_header(hotkey, media, axon_info.hotkey)
+
+        headers = {
+            "Content-Type": content_type,
+            "X-Media-Type": modality,
+            **headers,
+        }
+
+        if testnet_metadata:
+            testnet_headers = {f"X-Testnet-{k}": str(v) for k, v in testnet_metadata.items()}
+            headers.update(testnet_headers)
 
         async with session.post(
             url,
-            headers={
-                "Content-Type": content_type,
-                "X-Media-Type": modality,
-                **headers,
-            },
+            headers=headers,
             data=media,
             timeout=aiohttp.ClientTimeout(
                 total=total_timeout,

neurons/miner.py

@@ -140,6 +140,16 @@ def detect(self, media_tensor, modality):
         return probs
 
 
+def extract_testnet_metadata(headers):
+    headers = dict(headers)
+    testnet_metadata = {}
+    for key, value in headers.items():
+        if key.lower().startswith("x-testnet-"):
+            metadata_key = key[len("x-testnet-") :].lower()
+            testnet_metadata[metadata_key] = value
+    return testnet_metadata
+
+
 class Miner(BaseNeuron):
     neuron_type = NeuronType.MINER
     fast_api: FastAPIThreadedServer
@@ -189,6 +199,10 @@ async def detect_image(self, request: Request):
                 f"Unexpected content type: {content_type}, expected image/jpeg"
             )
 
+        testnet_metdata = extract_testnet_metadata(request.headers)
+        if len(testnet_metdata) > 0:
+            bt.logging.info(json.dumps(testnet_metdata, indent=2))
+
         try:
             image_array = np.array(Image.open(io.BytesIO(image_data)))
             image_tensor = torch.from_numpy(image_array).permute(2, 0, 1)
@@ -213,6 +227,11 @@ async def detect_video(self, request: Request):
             bt.logging.warning(
                 f"Unexpected content type: {content_type}, expected video/mp4 or video/mpeg"
             )
+
+        testnet_metdata = extract_testnet_metadata(request.headers)
+        if len(testnet_metdata) > 0:
+            bt.logging.info(json.dumps(testnet_metdata, indent=2))
+
         try:
             with tempfile.NamedTemporaryFile(suffix=".mp4") as temp_file:
                 temp_path = temp_file.name

neurons/validator.py

@@ -87,7 +87,7 @@ def init(self):
                 self.send_challenge_to_miners_on_interval,
                 self.update_compressed_cache_on_interval,
                 self.update_media_cache_on_interval,
-                self.start_new_wanbd_run_on_interval
+                self.start_new_wanbd_run_on_interval,
             ]
         )
 
@@ -218,6 +218,11 @@ async def send_challenge_to_miners_on_interval(self, block):
                         self.config.neuron.miner_total_timeout,
                         self.config.neuron.miner_connect_timeout,
                         self.config.neuron.miner_sock_connect_timeout,
+                        testnet_metadata=(
+                            {k: v for k, v in media_sample.items() if k != modality}
+                            if self.config.netuid != MAINNET_UID
+                            else {}
+                        ),
                     )
                 )
             if len(challenge_tasks) != 0:
@@ -357,6 +362,7 @@ async def _sample_media(self) -> Optional[Dict[str, Any]]:
             kwargs = {
                 "min_duration": self.config.challenge.min_clip_duration,
                 "max_duration": self.config.challenge.max_clip_duration,
+                "max_frames": self.config.challenge.max_frames,
             }
 
         try:
@@ -396,7 +402,7 @@ async def _sample_media(self) -> Optional[Dict[str, Any]]:
 
         if sample and sample.get(modality) is not None:
             bt.logging.debug("Augmenting Media")
-            augmented_media, _, _ = apply_random_augmentations(
+            augmented_media, aug_level, aug_params = apply_random_augmentations(
                 sample.get(modality),
                 (256, 256),
                 sample.get("mask_center", None),
@@ -407,6 +413,9 @@ async def _sample_media(self) -> Optional[Dict[str, Any]]:
                     "modality": modality,
                     "media_type": media_type,
                     "label": MediaType(media_type).int_value,
+                    "metadata": sample.get("metadata", {}),
+                    "augmentation_level": aug_level,
+                    "augmentation_params": aug_params
                 }
             )
             return sample