Post

There was an error while loading. Please reload this page .

review: use llama_context_type instead of llama_graph_type

llama: allow partial seq_rm for GDN models for speculative decoding

Currently speculative checkpoint needs to restart from a checkpoint after some draft tokens are not accepted, this leads to some wastage in running the target again. This PR adds the ability to rollback upto draft_max by storing the GDN intermediates.

Extend the gated delta net kernel to store intermediate states for partial rollback support on the Metal backend.

delta_net_base: use ggml_pad instead of new_tensor