Skip to content

Iterator Protocol

Jump to bottom
Roger Johansson edited this page Jan 14, 2026 · 1 revision

Iterator Protocol

This page documents how Asynkron.JsEngine implements JavaScript's iterator and iterable protocols, including built-in iterators, for-of loops, and the TC39 iterator helpers.

Protocol Overview

flowchart TB
    subgraph Iterable["Iterable Protocol"]
        SymIter["[Symbol.iterator]()"]
        Returns1["returns Iterator"]
    end
    
    subgraph Iterator["Iterator Protocol"]
        Next["next()"]
        Returns2["returns { value, done }"]
    end
    
    subgraph Result["IteratorResult"]
        Value["value: any"]
        Done["done: boolean"]
    end
    
    SymIter --> Returns1
    Returns1 --> Iterator
    Next --> Returns2
    Returns2 --> Result
Loading

Iterable Protocol: An object is iterable if it has a [Symbol.iterator] method that returns an iterator.

Iterator Protocol: An iterator has a next() method that returns { value, done } objects.

Core Iterator Types

IteratorResultObject

The result of calling next() on an iterator:

File: JsTypes/IteratorResultObject.cs

internal sealed class IteratorResultObject : IJsObjectLike, IAsJsValue, IJsSurfacedMutable
{
    // Singleton for exhausted iterators
    public static readonly IteratorResultObject DoneUndefined = new(JsValue.Undefined, true);
    
    private JsValue _value;
    private bool _done;
    
    public bool TryGetProperty(string name, out JsValue value)
    {
        switch (name)
        {
            case "value": value = _value; return true;
            case "done": value = _done ? JsValue.True : JsValue.False; return true;
            default: value = JsValue.Undefined; return false;
        }
    }
}

Pooling: Iterator results are pooled via IteratorResultObjectPool to minimize allocations.

JsIteratorBase

Base class for all built-in iterators:

File: JsTypes/JsIteratorBase.cs

public abstract class JsIteratorBase : IJsObjectLike, IAsJsValue, IPrototypeAccessorProvider
{
    protected bool _done;
    
    // All iterators are also iterable (return themselves)
    // Symbol.iterator returns `this`
}

Well-Known Symbols

File: Ast/Symbols.cs

public static class Symbols
{
    public static readonly JsSymbol Iterator = JsSymbol.For("Symbol.iterator");
    public static readonly JsSymbol AsyncIterator = JsSymbol.For("Symbol.asyncIterator");
    public static readonly JsSymbol ToStringTag = JsSymbol.For("Symbol.toStringTag");
    // ... other symbols
}

Built-in Iterators

Array Iterator

Files:

  • JsTypes/JsArrayIterator.cs
  • StdLib/ArrayIterator/ArrayIteratorPrototype.cs
  • StdLib/Array/ArrayPrototype.Iterators.cs
internal sealed class JsArrayIterator : JsIteratorBase
{
    internal enum ArrayIteratorKind { Entries, Keys, Values }
    
    private readonly JsArray _array;
    private readonly ArrayIteratorKind _kind;
    private int _index;
    
    internal JsValue Next()
    {
        if (_done) return IteratorResultObject.DoneUndefined.AsJsValue;
        
        if (_index >= _array.Length)
        {
            _done = true;
            return IteratorResultObject.DoneUndefined.AsJsValue;
        }
        
        var value = _kind switch
        {
            ArrayIteratorKind.Keys => JsValue.FromInt32(_index),
            ArrayIteratorKind.Values => _array.GetElement(_index),
            ArrayIteratorKind.Entries => CreateEntryPair(_index, _array.GetElement(_index)),
            _ => throw new InvalidOperationException()
        };
        
        _index++;
        return new IteratorResultObject(value, false).AsJsValue;
    }
}

Array Methods:

Method Returns
arr.values() Iterator over values (default)
arr.keys() Iterator over indices
arr.entries() Iterator over [index, value] pairs
arr[Symbol.iterator]() Same as values()

Map Iterator

File: JsTypes/JsMapIterator.cs

internal sealed class JsMapIterator : JsIteratorBase
{
    internal enum MapIteratorKind { Entries, Keys, Values }
    
    private readonly JsMap _map;
    private readonly IEnumerator<KeyValuePair<JsValue, JsValue>> _enumerator;
    private readonly MapIteratorKind _kind;
}

Map Methods:

Method Returns
map.values() Iterator over values
map.keys() Iterator over keys
map.entries() Iterator over [key, value] pairs (default)

Set Iterator

File: JsTypes/JsSetIterator.cs

internal sealed class JsSetIterator : JsIteratorBase
{
    internal enum SetIteratorKind { Entries, Values }
    
    private readonly JsSet _set;
    private readonly IEnumerator<JsValue> _enumerator;
    private readonly SetIteratorKind _kind;
}

String Iterator

Strings are iterable by code point (not UTF-16 code unit):

File: StdLib/Iteration/IterationHelper.cs

private static JsObject CreateStringIterator(string str)
{
    // Per ES spec 22.1.5.2.1 %StringIteratorPrototype%.next():
    // Strings are iterated by code point, not UTF-16 code unit.
    // Surrogate pairs should be returned as a single string.
    
    var index = 0;
    return CreateIterator(() =>
    {
        if (index >= str.Length) return (JsValue.Undefined, true);
        
        var cp = char.ConvertToUtf32(str, index);
        var result = char.ConvertFromUtf32(cp);
        index += result.Length;  // 1 for BMP, 2 for surrogate pair
        
        return (JsValue.FromString(result), false);
    });
}
// Surrogate pair example
const emoji = "😀";  // U+1F600, stored as surrogate pair
for (const char of emoji) {
    console.log(char);  // Logs "😀" once, not two surrogates
}

for-of Loop Implementation

IR Emission

File: Execution/Emitters/ForOfEmitter.cs

flowchart TB
    subgraph ForOf["for (const x of iterable)"]
        Init["IteratorInit\n(get iterator)"]
        TryEnter["EnterTry"]
        Loop["Loop Start"]
        MoveNext["IteratorMoveNext"]
        Check{done?}
        Body["Loop Body"]
        Finally["Finally Block"]
        Close["IteratorClose"]
    end
    
    Init --> TryEnter
    TryEnter --> Loop
    Loop --> MoveNext
    MoveNext --> Check
    Check -->|no| Body
    Body --> Loop
    Check -->|yes| Finally
    Finally --> Close
Loading

The for-of loop is lowered to IR instructions:

// ForOfEmitter.cs - Simplified
public void Emit(ForOfStatement node)
{
    // 1. Get iterator from iterable
    EmitIteratorInit(node.Right, isAsync: false);
    
    // 2. Wrap in try/finally for proper cleanup
    var tryIndex = EnterTry();
    
    // 3. Loop: call next() and check done
    var loopLabel = CreateLabel();
    EmitIteratorMoveNext();
    BranchIfDone(exitLabel);
    
    // 4. Bind value to loop variable
    EmitBinding(node.Left);
    
    // 5. Execute body
    EmitBody(node.Body);
    Branch(loopLabel);
    
    // 6. Finally: close iterator
    EmitFinally();
    EmitIteratorClose();
    EndFinally();
}

Runtime Handlers

File: Ast/TypedAstEvaluator.ExecutionPlanRunner.Handlers.Iterators.cs

private void HandleIteratorInit(IteratorInitInstruction instruction)
{
    var iterable = GetSlotValue(instruction.IterableSlot);
    
    // Get Symbol.iterator method
    if (!TryGetIteratorFromProtocols(iterable, out var iterator, out var enumerator))
    {
        throw ThrowTypeError("Value is not iterable", context);
    }
    
    // Store iterator state
    SetSlotValue(instruction.IteratorSlot, iterator);
    SetSlotValue(instruction.EnumeratorSlot, enumerator);
}

private void HandleIteratorMoveNext(IteratorMoveNextInstruction instruction)
{
    var iterator = GetSlotValue(instruction.IteratorSlot);
    var next = iterator.GetProperty("next");
    var result = Call(next, iterator, []);
    
    var done = result.GetProperty("done").IsTrue;
    var value = result.GetProperty("value");
    
    SetSlotValue(instruction.ValueSlot, value);
    SetSlotValue(instruction.DoneSlot, done);
}

private void HandleIteratorClose(IteratorCloseInstruction instruction)
{
    var iterator = GetSlotValue(instruction.IteratorSlot);
    
    // Call return() if it exists
    if (iterator.TryGetProperty("return", out var returnMethod))
    {
        Call(returnMethod, iterator, []);
    }
}

IteratorClose Semantics

Per ECMA-262 §7.4.7, iterators should be closed when iteration ends early:

File: Ast/JsObjectExtensions.cs

internal static void IteratorClose(this IJsObjectLike iterator, EvaluationContext context, 
    bool preserveExistingThrow = false)
{
    // If return() method exists, call it
    if (iterator.TryGetProperty("return", out var returnMethod) && returnMethod.IsFunction)
    {
        var innerResult = Call(returnMethod, iterator, []);
        
        // If preserveExistingThrow, restore the original exception after cleanup
    }
}

This ensures cleanup happens on:

  • break statements
  • return statements inside loops
  • Thrown exceptions

Spread Operator

The spread operator uses the iterator protocol:

File: Ast/TypedAstEvaluator.cs

// SpreadElement runtime semantics (ECMA-262 §12.2.5.2)
private static IEnumerable<JsValue> EnumerateSpread(JsValue value, EvaluationContext context)
{
    if (!TryGetIteratorForDestructuring(value, context, out var iterator, out var enumerator))
    {
        throw ThrowTypeError("Value is not iterable", context);
    }
    
    var iteratorRecord = new ArrayPatternIterator(iterator, enumerator);
    while (true)
    {
        var (item, done) = iteratorRecord.Next(context);
        if (done) yield break;
        yield return item;
    }
}
// Spread uses iterator protocol
const arr = [1, 2, 3];
const copy = [...arr];           // Array iterator
const str = [...'abc'];          // String iterator: ['a', 'b', 'c']
const map = [...new Map([[1, 'a']])];  // Map iterator

Iterator Helpers (TC39 Proposal)

Asynkron.JsEngine implements the TC39 Iterator Helpers proposal:

File: StdLib/Iterator/IteratorPrototype.cs

Transformation Methods

Method Description
map(fn) Transform each value
filter(fn) Keep values where fn returns true
take(n) Take first n values
drop(n) Skip first n values
flatMap(fn) Map and flatten one level 
[JsHostMethod("map", Length = 1d)]
public JsValue Map(JsValue thisValue, IReadOnlyList<JsValue> args)
{
    var iterator = GetIterator(thisValue);
    var mapper = args.GetArgument(0);
    
    return CreateIterator(() =>
    {
        var (value, done) = iterator.Next();
        if (done) return (JsValue.Undefined, true);
        
        var mapped = Call(mapper, JsValue.Undefined, [value]);
        return (mapped, false);
    });
}

[JsHostMethod("filter", Length = 1d)]
public JsValue Filter(JsValue thisValue, IReadOnlyList<JsValue> args)
{
    var iterator = GetIterator(thisValue);
    var predicate = args.GetArgument(0);
    
    return CreateIterator(() =>
    {
        while (true)
        {
            var (value, done) = iterator.Next();
            if (done) return (JsValue.Undefined, true);
            
            if (Call(predicate, JsValue.Undefined, [value]).IsTrue)
            {
                return (value, false);
            }
        }
    });
}

[JsHostMethod("take", Length = 1d)]
public JsValue Take(JsValue thisValue, IReadOnlyList<JsValue> args)
{
    var iterator = GetIterator(thisValue);
    var limit = (int)ToNumber(args.GetArgument(0));
    var taken = 0;
    
    return CreateIterator(() =>
    {
        if (taken >= limit) return (JsValue.Undefined, true);
        
        var (value, done) = iterator.Next();
        if (done) return (JsValue.Undefined, true);
        
        taken++;
        return (value, false);
    });
}

Consumption Methods

Method Description
reduce(fn, init) Reduce to single value
toArray() Collect into array
forEach(fn) Execute callback for each
some(fn) True if any match
every(fn) True if all match
find(fn) Find first matching 
[JsHostMethod("toArray", Length = 0d)]
public JsValue ToArray(JsValue thisValue, IReadOnlyList<JsValue> _)
{
    var iterator = GetIterator(thisValue);
    var result = new List<JsValue>();
    
    while (true)
    {
        var (value, done) = iterator.Next();
        if (done) break;
        result.Add(value);
    }
    
    return JsArray.From(result);
}

[JsHostMethod("reduce", Length = 1d)]
public JsValue Reduce(JsValue thisValue, IReadOnlyList<JsValue> args)
{
    var iterator = GetIterator(thisValue);
    var reducer = args.GetArgument(0);
    var accumulator = args.Count > 1 ? args[1] : GetFirstValue(iterator);
    
    while (true)
    {
        var (value, done) = iterator.Next();
        if (done) return accumulator;
        
        accumulator = Call(reducer, JsValue.Undefined, [accumulator, value]);
    }
}

Iterator.from() and Iterator.concat()

File: StdLib/Iterator/IteratorConstructor.cs

[JsHostMethod("from", Length = 1d)]
public JsValue From(JsValue _, IReadOnlyList<JsValue> args)
{
    var value = args.GetArgument(0);
    
    // If already an iterator, return as-is
    if (value.TryGetProperty(Symbols.Iterator, out var iterMethod))
    {
        return Call(iterMethod, value, []);
    }
    
    throw ThrowTypeError("Value is not iterable", context);
}

[JsHostMethod("concat", Length = 0d)]
public JsValue Concat(JsValue _, IReadOnlyList<JsValue> args)
{
    var iterables = args.ToArray();
    var currentIndex = 0;
    IEnumerator<JsValue>? currentIterator = null;
    
    return CreateIterator(() =>
    {
        while (true)
        {
            if (currentIterator != null && currentIterator.MoveNext())
            {
                return (currentIterator.Current, false);
            }
            
            if (currentIndex >= iterables.Length)
            {
                return (JsValue.Undefined, true);
            }
            
            currentIterator = GetIterator(iterables[currentIndex++]);
        }
    });
}

Symbol.dispose

Iterators support the using declaration (TC39 Explicit Resource Management):

[JsHostMethod(SymbolKeys.Dispose, Length = 0d)]
public JsValue Dispose(JsValue thisValue, IReadOnlyList<JsValue> _)
{
    // Close the iterator
    IteratorClose(thisValue);
    return JsValue.Undefined;
}
// Using declaration auto-closes iterator
{
    using iter = arr.values().take(5);
    for (const x of iter) {
        console.log(x);
    }
}  // iter[Symbol.dispose]() called here

Async Iteration

Symbol.asyncIterator

For for await...of loops:

File: StdLib/Iteration/IterationHelper.cs

public static JsValue GetAsyncIterator(JsValue value, EvaluationContext context)
{
    // First check Symbol.asyncIterator
    if (value.TryGetProperty(Symbols.AsyncIterator, out var asyncIterMethod))
    {
        return Call(asyncIterMethod, value, []);
    }
    
    // Fall back to Symbol.iterator and wrap in async
    if (value.TryGetProperty(Symbols.Iterator, out var iterMethod))
    {
        var syncIterator = Call(iterMethod, value, []);
        return WrapSyncIteratorAsAsync(syncIterator);
    }
    
    throw ThrowTypeError("Value is not async iterable", context);
}

for await...of

flowchart TB
    subgraph ForAwaitOf["for await (const x of asyncIterable)"]
        Init["Get async iterator"]
        Loop["Loop Start"]
        AwaitNext["await iterator.next()"]
        Check{done?}
        Body["Body"]
        AwaitClose["await iterator.return()"]
    end
    
    Init --> Loop
    Loop --> AwaitNext
    AwaitNext --> Check
    Check -->|no| Body
    Body --> Loop
    Check -->|yes| AwaitClose
Loading

Performance Optimizations

Object Pooling

Iterator results are pooled to reduce GC pressure:

internal static class IteratorResultObjectPool
{
    private static readonly ObjectPool<IteratorResultObject> Pool = new(() => new());
    
    public static IteratorResultObject Rent(JsValue value, bool done)
    {
        var obj = Pool.Get();
        obj.Initialize(value, done);
        return obj;
    }
    
    public static void Return(IteratorResultObject obj)
    {
        if (!obj.IsCaptured)
        {
            Pool.Return(obj);
        }
    }
}

Fast Path for Simple Loops

File: Ast/IteratorDriverPlanExtensions.cs

The iterator driver has optimizations for common patterns:

// Fast path: simple accumulator pattern
// for (const x of arr) sum += x;
if (IsSimpleAccumulatorPattern(body))
{
    return ExecuteFastAccumulator(iterator, body);
}

// Fast path: direct slot binding
// for (const x of arr) { ... }  // x is simple identifier
if (binding is IdentifierPattern)
{
    // Direct slot write, no destructuring overhead
}

Dual Path Execution

The engine maintains both:

  1. Iterator protocol path (next/done/value objects)
  2. IEnumerator path for internal use
// Internal enumeration is faster
internal static IEnumerable<JsValue> EnumerateValues(JsValue iterable)
{
    if (iterable.TryGetObject<JsArray>(out var arr))
    {
        // Fast path: direct array access
        foreach (var item in arr.Items)
        {
            yield return item;
        }
    }
    else
    {
        // Protocol path
        var iterator = GetIterator(iterable);
        // ... standard protocol
    }
}

Key Files

Category File Path
IteratorResult src/Asynkron.JsEngine/JsTypes/IteratorResultObject.cs
Base Class src/Asynkron.JsEngine/JsTypes/JsIteratorBase.cs
Array Iterator src/Asynkron.JsEngine/JsTypes/JsArrayIterator.cs
Map Iterator src/Asynkron.JsEngine/JsTypes/JsMapIterator.cs
Set Iterator src/Asynkron.JsEngine/JsTypes/JsSetIterator.cs
Symbols src/Asynkron.JsEngine/Ast/Symbols.cs
Iteration Helpers src/Asynkron.JsEngine/StdLib/Iteration/IterationHelper.cs
Iterator Prototype src/Asynkron.JsEngine/StdLib/Iterator/IteratorPrototype.cs
Iterator Constructor src/Asynkron.JsEngine/StdLib/Iterator/IteratorConstructor.cs
for-of Emitter src/Asynkron.JsEngine/Execution/Emitters/ForOfEmitter.cs
Iterator Handlers src/Asynkron.JsEngine/Ast/TypedAstEvaluator.ExecutionPlanRunner.Handlers.Iterators.cs
Iterator Driver src/Asynkron.JsEngine/Execution/IteratorDriverPlan.cs
Iterator Close src/Asynkron.JsEngine/Ast/JsObjectExtensions.cs
Result Pool src/Asynkron.JsEngine/Pooling/IteratorResultObjectPool.cs

See Also

Clone this wiki locally