Ray Releases

Highlights Ray Serve LLM: In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the LLMRouter ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress replica ( #64642 , #64920 , #64949 , #65010 , #65095 ). KV cache and token aware routing is also aware of CPU KV caches, so offloaded KV cache blocks count toward a replica's cache hit ( #65063 ). Ray Core: We enabled the capability to offload task events from. With RAY_enable_task_events_to_dashboard_head on, the task event buffer is replaced by the ray event recorder, events are exported from the aggregator agent to a task events head that keeps an in-memory store, and the state APIs and ray.timeline read from it ( #64835 , #65028 , #65123 , #65160 , #65218 ). Enabling the feature removes task event ingestion and serving from the GCS hot path. Ray Data: We’ve added Databricks integrations for writing to DeltaLake, with Catalog support. We’ve also shipped a new shuffle v2 backend, featuring improved performance for joins and aggregations. Sandboxing: We've also added experimental Ray Sandbox, which runs task and actor code under gVisor and can run Docker-built images directly ( #64964 , #65570 ). TPU Support: Ray Train adds support for TorchTPU backend ( #64796 ), and Ray Core adds SubslicePlacementGroup for gang scheduling on TPU subslices, single-host TPU support in SlicePlacementGroup , and resource accounting for tpu7x and multi-core chips ( #64578 , #64079 , #64058 ). This lets TPU slices and subslices be reserved and trained on without external gang-scheduling glue. Ray Data 🎉 New Features Add Dataset.with_columns for multi-column expression projection ( #63858 ) Add write_delta for Delta Lake, with catalog support ( #64923 , #65079 ) Add Torch inference API ( #65157 ) Promote hash shuffle v2 to a selectable shuffle strategy, with aggregation support, vectorized aggregation, and block splitting during aggregation ( #64953 , #64652 , #64956 , #65329 , #64897 ) Add ignore_missing_paths and skip_paths to read_parquet on DatasourceV2 ( #65118 ) Add delta_timestamps (temporal windows) to read_lerobot ( #64877 ) Collect cluster usage metrics by sampling in background threads during execution ( #64686 ) Tolerate actor deaths during init via DataContext.max_consecutive_actor_init_deaths ( #64846 ) Expose RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES in DataContext ( #65103 ) 💫 Enhancements Generate sortable, collision-resistant dataset IDs ( #65075 ) Push Limit into ReadFiles when it sits directly on it, and extract FileIndexer.list_file_infos ( #65167 , #65168 ) Use BlockRefCounter for object store memory estimation and remove BlockRefCounter.clear() ( #64456 , #64521 ) Fail execution if no operator makes progress within a timeout, instead of hanging ( #65349 ) Improve BatchIterator and iter_torch_batches , and allow a custom collate_fn with a custom device ( #64994 , #64967 ) Skip downstream-capacity backpressure for eligible materializers ( #64844 ) Shard exact download partitioning ( #64066 ) Bump the sort_reduce memory multiplier to 3x and stop using estimated_input_blocks as the shuffle partition count ( #65176 , #65296 , #65335 ) Disable cluster autoscaling when PlacementGroupSchedulingStrategy is in use ( #64417 ) Propagate the operator name to shuffle tasks ( #64802 ) Rename reports_custom_op_stats to should_report_custom_op_stats across all MapTransformFn variants ( #64461 , #64515 ) Rename allocated to reserved and add type aliases in the autoscaling coordinator ( #64997 , #65096 ) Deprecate ray_remote_args_fn and Dataset.zip ( #64963 , #65111 ) 🔨 Fixes Fix an RCE where read_lance or nested pickle objects could execute arbitrary code ( #64881 ) Stop converting Arrow null columns to null[pyarrow] in to_pandas ( #65187 ) Fix iter_torch_batches device resolution and typing ( #65059 , #64947 ) Propagate isolate_read_workers to DatasourceV2 ( #65191 ) Avoid signalling epoch end for failed attempts ( #65082 ) Add tf-keras to the text_embedding pip packages ( #64889 ) 📖 Documentation Fix the stale use_datasource_v2 docstring default and an incorrect default_map_logical_memory_enabled reference ( #65155 , #65091 ) Ray Serve 🎉 New Features Configurable status code and Retry-After header for backpressure rejections ( #65193 , #65319 ) Add dependency-ordered shutdown for deployments ( #64922 ) Add an optional tie-break key to best-fit node scheduling ( #64914 ) Scale ingress request router replicas per proxy node ( #64724 ) 💫 Enhancements Reconcile health checks from a dirty set, sweeping RUNNING replicas round-robin instead of every tick ( #64690 ) Gate the rank-consistency check on replica membership changes ( #64911 ) Read the multiplex marker statically so probing cannot initialize handles ( #65064 ) Type CreatePlacementGroupRequest.runtime_env as a dict ( #64892 ) Add a deployment-state accessor for testing ( #64790 ) 🔨 Fixes Fix the Serve replica ASGIService bypassing token authentication ( #65189 ) Fix the proxy update loop getting stuck when a proxy's node is removed ( #64403 ) Ray Train 🎉 New Features Ray Train Integrates with TorchTPU backend ( #64796 ) Add a public preemption API and a controller PreemptingState ( #64360 ) Add data ingest metrics to the Train dashboard ( #64523 ) 💫 Enhancements Make Train V1 and V2 use the autoscaling coordinator ( #64824 ) Expand the contains_tensor check and add a serialization check for the results return value ( #64930 ) Pin PlacementGroupCleaner to the head node ( #64705 ) 🔨 Fixes Fix Torch environment setup for V1 worker groups ( #65005 ) Ray Tune 🔨 Fixes Fix HyperOptSearch dropping tune.choice categories that are constant dicts ( #64537 ) Deflake test_multi_trial_reuse_with_failing and decide test_experiment_restore completion from measured progress ( #64526 , #65212 ) Ray LLM 🎉 New Features KV-cache-aware routing: move tokenization into the LLMRouter ingress replica, decide KV/token routing there, broadcast KV lifecycle events to all ingress replicas, and make selection and reservation atomic ( #64642 , #64920 , #64949 , #65010 ) Enable KV cache offloading, make KV routing aware of CPU KV caches, and transmit tokens out-of-band so the engine skips tokenization ( #65063 , #65095 ) Add a KV cache offload/reload dashboard and a Ray Serve LLM SGLang metrics dashboard ( #65122 , #64797 ) 💫 Enhancements Upgrade to vLLM 0.26.0 ( #65045 ) Route direct-streaming ingress to the co-located router ( #64489 ) Reuse vLLM's resolved HF config in apply_checkpoint_info ( #62962 ) Preserve mapping-valued vLLM frontend arguments ( #65146 ) 📖 Documentation Document loading models from Azure storage, including az:// Blob streaming with RunAI Streamer ( #64819 , #64825 ) Ray RLlib 🔨 Fixes Fix TQC critic divergence by stopping actor-loss gradients from leaking into the critics ( #65125 ) Fix squashed-Gaussian log-prob corruption for saturated policies ( #65036 ) Use the target_qf_twin head in IQL target prediction ( #64932 ) Properly enforce use_kl_loss in the PPO Torch and TF policies ( #61562 ) Fix a KeyError in the multi-agent module-to-env connector ( #64803 ) Fix the API-doc consistency check for the new reverse/dedup policies, and deflake the test_env_runner callback-count tests ( #64807 , #64989 ) Ray Core 🎉 New Features Ray Sandbox (experimental): run task and actor code under gVisor, and run Docker-built images out of the box ( #64964 , #65397 , #65570 , #65622 ) Move task events out of the GCS: the ray event recorder replaces the task event buffer, the aggregator agent exports to a task events head with an in-memory store, and the state APIs and ray.timeline are rerouted to it, with reconciliation on worker death and job completion ( #64835 , #65028 , #65057 , #65123 , #65141 , #65160 , #65218 , #65247 , #65288 ) Add SubslicePlacementGroup for gang scheduling on TPU subslices, support single-host TPUs in SlicePlacementGroup , and add a per_slice_pgs parameter ( #64578 , #64079 , #64072 ) Introduce a native, lightweight C++ leader election client for active-passive GCS ( #63773 ) Add Apple silicon GPU ( mps ) support and an Intel GPU ZE_AFFINITY_MASK mapping ( #38464 , #64440 ) Add worker lifecycle events to the events export pipeline ( #64887 ) Add GPU UUID to the labels of GPU metrics ( #65113 ) Enable resource accounting for tpu7x and multi-core chips, add gb200 / gb300 accelerator constants, and add TTNPU custom accelerator resources ( #64058 , #65009 , #61554 ) Support cross-device transfers in RDT NIXL ( #64815 ) 💫 Enhancements Publish node death before persisting it and drop RocksDB soft durability (REP-64) ( #64702 ) Make RedisContext::Connect non-fatal on connection failure ( #64299 ) Move the pending resource load pull off the GCS main io_context ( #65024 ) Subscribe only to the specific owner worker's death for generator backpressure, and only when actor-level backpressure is enabled ( #65195 , #65136 ) Evict dead actors from ActorPool instead of recycling them ( #64646 ) Rename label_domain to topology strategy in the scheduling policy ( #64384 ) Warn on use of the deprecated dynamic generator ( #64749 ) Add object resolution debug logging for lineage reconstruction ( #64853 ) Block only on the CUDA stream used to create tensors in RDT ( #64823 ) Preserve StateSchema column order in filter_fields ( #65052 ) Prepare StreamResponse on an empty log stream ( #62296 ) Refine ObjectRefStreamEndOfStreamError from _get_next_ref_n ( #64602 ) Drop a redundant FunctionDescriptor rebuild in CallSiteString ( #64874 ) Upgrade bundled dependencies: log4j 2.25.4, jackson-databind 2.18.8 ( CVE-2026-54512 , CVE-2026-54513 ), gson 2.11.0, aiohttp, idna, and azure ( #64269 , #64575 , #64273 , #65131 , #64056 , #65046 ) 🔨 Fixes Give canceled_tasks_ its own mutex to break a lock-order cycle ( #65393 , #65620 ) Fix a GIL/mutex deadlock in actor-level backpressure with sync and async streaming generators ( #64896 ) Fix a deadlock between metric registration and collect() in OpenTelemetryMetricRecorder ( #64946 ) Report shutdown from check_signals instead of exiting the process ( #65184 , #65400 ) Fix a Python 3.14 async-actor memory leak by re-anchoring stack protection to fiber stacks ( #64772 ) Keep only the latest object-location pubsub snapshot to fix an owner memory leak ( #65133 ) Fix a spurious OwnerDiedError during graceful raylet shutdown ( #64899 ) Fail ray.get on refs from a non-restartable streaming generator when those objects are lost ( #64756 ) Serve the spilled copy when Push hits a stale local_objects_ mirror ( #64916 ) Fix spill_manager_objects_bytes reporting the restored object count instead of restored bytes ( #65013 ) Forward node-pinned actors to the pinned node in GCS actor scheduling ( #64951 ) Fix a leaked named actor and name conflict when registration times out ( #64948 ) Fix a set-before-register race in RDTManager that could SIGSEGV ( #64558 ) Fix task log info fields in lifecycle events ( #65190 ) Guard Status::operator<< against an OK status, and fix UB in StatusOr swap and assignment on error-state operands ( #64983 , #64799 ) Compare all ResourceRequest fields in operator== ( #64838 ) Verify VFIO groups are backed by Google TPU PCI devices, and use POSIX paths for VFIO sysfs vendor checks on Windows ( #65105 , #65182 ) 📖 Documentation Document the embedded RocksDB GCS backend ( #64731 ) Clarify actor class state isolation and the Ray Core walkthrough benchmark setup ( #64597 , #64529 ) Document how to set up placement group topology strategy on Kubernetes ( #64117 ) Dashboard 🎉 New Features Capture Kubernetes Pod events in the Ray Dashboard head ( #63937 ) 💫 Enhancements Hide the GPU and GRAM columns when no GPUs are present ( #64567 ) Update axios to ^1.18.0 ( #65130 ) 🔨 Fixes Fix the profiling status check so it works behind a reverse proxy ( #65126 ) Fix a dashboard startup crash from an unguarded kubernetes import ( #64962 ) 📖 Documentation Add a user guide for Kubernetes events in the Ray Dashboard ( #64734 ) Ray Wheels and Images 🎉 New Features Enable Windows py3.13 and py3.14 wheel builds ( #64970 ) Publish arm64 Ray LLM images ( #65002 ) Add a ray-torch release test image (py3.14, cu12.8) and a hello_world_py314 smoke release test ( #65114 , #64857 ) 💫 Enhancements Bump the Anyscale CLI to 0.26.105 and bake pybase64 into the ML release-test image ( #64980 , #64791 ) Recompile dependency lock files on dependabot PRs, and regenerate the ray-torch py3.14 lock for aiohttp 3.14.3 ( #65056 , #65343 , #65378 ) Pin grpcio to 1.75.0 in test deps to avoid the grpc.aio performance regression ( #65112 ) Give ray-wheel-minimal-build a distinct wanda image name ( #65318 , #65383 ) Move cu130 job tests off g4dn.4xlarge to g6.4xlarge ( #65213 ) Bump the version to 2.58.0 and publish 2.56.0 perf metrics ( #65252 , #64196 ) 🔨 Fixes Fix org_lzma_lzma download failures by using the SourceForge redirector ( #64906 ) Install the data CI depset after conda ffmpeg so removed packages are restored ( #65334 , #65341 ) Drop -Wl,-pie from the vendored RocksDB WITH_TSAN link flags ( #64917 ) Floor peft>=0.18 for transformers 5.x in the huggingface_accelerate release test ( #65062 ) Documentation Add initial documentation for Ray sandboxing ( #65503 , #65573 ) Update the Ray History Server docs for RAY_ROOT_DIR -> STORAGE_ROOT_DIR and use a RayJob sample YAML ( #65139 , #65441 , #65510 , #65505 , #65531 ) Add a contributor guide for editing and managing Python dependencies ( #63547 ) Clarify the API deprecation policy ( #65093 ) Document the safe-to-evict annotation for the Ray head Pod, and autoscaler v1 restartPolicy behavior ( #64907 , #64900 ) Update the Gaudi tutorials and examples to the latest versions ( #58861 ) Correct the TLS verification comments and the metric cardinality default comments ( #61977 , #64478 ) Route API reference pages and autodoc machinery to the API-surface checks, ignore inherited API annotations, walk ray.data.llm as its own head module, and reserve the doc tag for doc validation ( #64812 , #65196 , #65040 , #64775 , #65208 ) Scope API signature bold weight to the object name, and reclassify sphinx unknown-document and docutils-inline-markup as judgment ( #64933 , #64839 ) Fix minor typos, bump the docs template build-id pins, and add Douglas Strodtman to the committer list ( #64864 , #65041 , #65092 ) Thanks Many thanks to all those who contributed to this release! @bveeramani , @yjaw , @sampan-s-nayak , @spencer-p , @iaalm , @risjai , @kahlun , @liulehui , @eicherseiji , @jhasm , @ronny-anyscale , @KuongB , @Ranoobaba , @Sparks0219 , @alimaazamat , @yuhuan130 , @jeffreywang88 , @AarryaSaraf , @johntomcat7408-cmyk , @tqKhanh1712 , @praneethkaturi , @elliot-barn , @tanmayrauth , @skpark-rh , @YashwanthRanjanSingaravel , @Yicheng-Lu-llll , @pseudo-rnd-thoughts , @nadongjun , @vineethsaivs , @owenowenisme , @saitejabandaru-in , @RocMarshal , @karticam , @win5923 , @Hyunoh-Yeo , @dragongu , @YoyinZyc , @chiayi , @HrushiYadav , @martinlhw , @sai-miduthuri , @dstrodtman , @JasonLi1909 , @2uchan , @mukktinaadh , @coqian , @ayushk7102 , @LuciferYang , @NripeshN , @MortalHappiness , @verma-divyanshu-git , @ans9868 , @johntaylor-cell , @prasad-anyscale , @edoakes , @ShockYoungCHN , @fscnick , @JiangJiaWei1103 , @subpath , @shivamsingh-007 , @RinZ27 , @richabanker , @iamjustinhsu , @malsbat , @andrewsykim , @ryanaoleary , @vinay7373 , @Kunchd , @petern48 , @kyuds , @rueian , @rayhhome , @xyuzh , @hao-aaron , @CaiZhanqi , @kalyanamdewri , @abhishekverma-ray , @khluu , @nh-atuan , @odncode , @robertnishihara , @machichima , @Arkit003 , @goutamvenkat-anyscale , @ArturNiederfahrenhorst , @Myasuka

Highlights Ray Data : We added fixes for several to_pandas regressions introduced in 2.56: an opt-out flag ( RAY_DATA_ENABLE_ARROW_BACKED_PANDAS_CONVERSION ) for Arrow-backed conversion, an int64/ double[pyarrow] overflow crash on concatenation, and a TensorDtype.__from_arrow__ crash on empty tensor columns ( #64793 , #64794 ). Ray Core : We added early detection for system-slice memory pressure: the memory monitor now snapshots the user and system cgroup slices together and logs an error when the system slice exceeds reserved system memory, warning users to raise --system-reserved-memory before it causes node deaths ( #64492 ). Ray Serve : We added protobuf 7 compatibility and a routing fix for LLM direct streaming, so body-aware routers like PrefixCacheAffinityRouter no longer hang when RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING=1 ( #64592 , #64488 ). Ray Data 🔨 Fixes Fixed two Arrow-backed to_pandas regressions: added DataContext.enable_arrow_backed_pandas_conversion as an opt-out, and reconciled divergent numeric column types before concatenation to avoid int64/ double[pyarrow] overflow crashes ( #64793 , #64768 ). Fixed a TensorDtype.__from_arrow__ crash on zero-size tensor elements by using an explicit row count instead of numpy's -1 dimension inference ( #64794 , #64767 ). Fixed a crash in hash partition caused by read-only hash arrays ( #64584 , #64552 , #64559 ). Nullified _input_dependencies in _get_args so exporting operator args no longer triggers an exponential sanitize_for_struct call chain over fused operators ( #64412 , #64316 ). Ray Serve 🔨 Fixes Added protobuf >=7 compatibility to _proto_to_dict by binding to FieldDescriptor.is_repeated when the deprecated label attribute is absent ( #64592 , #64362 ). Ray LLM 🔨 Fixes Fixed direct-streaming routing for body-aware routers: the ingress now parses the raw request body into a SimpleNamespace over routing-key fields ( messages , prompt ) so choose_replica receives the message body instead of raw bytes ( #64488 , #64328 , #64326 ). Ray RLlib 🔨 Fixes Upgraded the ONNX example from the retired MobileNet v1 to MobileNet v3 via torchvision and pinned onnxscript in the GPU/ml-build CI dep locks, fixing ONNX export failures ( #64591 , #64028 , #64031 , #64590 , #64033 ). Ray Core 💫 Enhancements The threshold memory monitor now snapshots both the user and system cgroup slices and logs an error when system-slice usage exceeds the reserved system memory, prompting users to raise --system-reserved-memory ( #64492 ). 🔨 Fixes Disabled NCCL cuMem host buffer registration in CI pytests to stabilize GPU test runs ( #64580 , #64146 ). Documentation 📖 Documentation Repointed the ASHA Tune example links to the renamed README document, fixing the fail_on_warning ReadTheDocs build on the release line ( #64761 , #64630 ). Updated the Python 3.10 CPU pip freeze dependency list for the Ray 2.56.0 release ( #64447 , #64357 ). Thanks Many thanks to all those who contributed to this release! @eicherseiji , @ryankert01 , @bveeramani , @elliot-barn , @iamjustinhsu , @Sparks0219 , @owenowenisme , @dstrodtman , @marwan116

Highlights Ray Data: In this Ray release, we've enabled DataSourceV2 by default ( #64821 ), so read_parquet and friends use the new scan/listing infrastructure with row-group-aware chunking and predicate splitting. Hash Shuffle V2 eliminates the aggregator actor pool. V1 had to provision that pool up front from an estimate of the input size, and its actors accumulated partition shards in actor heap memory, invisible to Ray and unspillable, until finalization. V2 replaces it with two stateless task-based operators, ShuffleMapOp --> ShuffleReduceOp, that pass shards through the object store, so intermediate state spills under pressure and no capacity has to be reserved in advance. The map/reduce barrier itself remains in both designs. HashShuffleV2 supports join ( #63598 , #64538 , #64687 ). This lets shuffles reuse standard map/reduce scheduling, backpressure, and resource accounting. Ray Serve: The HAProxy ingress is now distributed as the ray-haproxy PyPI package instead of being compiled into images, and it is the default HAProxy binary ( #64141 , #64163 , #64164 ). We've also added gRPC support to the HAProxy direct-ingress path, including streaming, metrics, and custom request IDs ( #63735 , #64310 , #64166 , #64112 ). For Ray Serve LLM, we've added experimental KV-cache-aware request routing that tracks replica KV state through an event plane, tokenizes before routing, and routes on prefill/decode token load ( #64084 , #64085 , #64097 , #64224 , #64327 , #64400 ). KV cache-aware routing’s complete support will land in 2.58. Ray Core: We've added an embedded RocksDB storage backend for GCS fault tolerance (REP-64), selectable with RAY_gcs_storage=rocksdb and RAY_gcs_storage_path ( #63657 ). GCS fault tolerance no longer requires an external Redis instance. We've also added a public API for topology-aware scheduling ( #63479 , #63740 ). Ray Data 🎉 New Features Enable DataSourceV2 by default via DataContext.use_datasource_v2 ( #64821 ) New task-based hash shuffle v2 ( ShuffleMapOp → ShuffleReduceOp ) with join , multi-input reduce, downstream map fusion, and reducer remote args, behind an env flag ( #63598 , #64538 , #64687 , #64438 , #64302 , #64532 , #64481 ) Add a Catalog abstraction with a UnityCatalog implementation that can be passed to read_* , and Unity Catalog write support for Parquet and Iceberg ( #64193 , #64519 ) Add read_zarr for Zarr datasets ( #63003 ) and read_lerobot for LeRobot v3 datasets ( #63821 ) Add PushdownCountFiles optimization to answer count() from Parquet footers ( #64763 ) Add common subexpression elimination to the expression optimizer ( #63974 ) Add GPU support for Aggregate ( #63708 ) Make dataset iteration metrics queryable per split ( #64608 ) Add custom operator stats to capture worker-side metrics during task execution ( #64221 ) Refactor usage collection into an extensible UsageCallback ( #64500 ) Export from_blocks from ray.data ( #64127 ) Add support for rapidsmpf-26.4.0 ( #64324 ) 💫 Enhancements Hide Ray Data internal frames from user-code error tracebacks ( #64587 ) Run projection/predicate pushdown before limit pushdown, block optimization of non-deterministic expressions, and make fuse checks consistent between the logical and physical optimizers ( #64651 , #64165 , #63936 ) Add a two-mode block metadata fetch behind a single MetadataFetcher interface ( #64378 ) Use work stealing for file listing and a dynamic work queue for traversals ( #64675 , #64388 ) Speed up ExecutionResources and the reservation/budget loops on the scheduling hot path ( #63964 ) Fix O(n^2) schema reconciliation in unify_schemas and avoid per-column Series materialization in tensor-column casting ( #64555 , #64038 ) Decouple collate and memory pinning in iter_torch_batches , add per-stage training-thread blocking attribution, bound in-flight iter_threaded items, and finalize after reordering under preserve_order ( #64653 , #64183 , #64219 , #64282 ) Add default logical memory for map operators, wire the materializing-op filter into OpResourceAllocator budgeting, and move estimate_object_store_usage into the physical op ( #63814 , #63665 , #63961 ) Decrease the downstream backpressure ratio to 2.0 ( #64352 ) Route native S3FileSystem downloads to the PyArrow threaded path ( #64089 ) Size hash-shuffle aggregators via bounded online sampling and add a timeout to the batched ray.get when fetching partitions ( #63929 , #64256 ) Replace manual block reference accounting with Ray Core out-of-scope object callbacks ( #64011 , #64157 , #64191 ) Default read_numpy to allow_pickle=False and make it manually configurable ( #64684 ) Make write_lance(mode=CREATE) error instead of silently overwriting ( #64364 ) Expand DistributionTracker with merge() and p25/p75, and add dead node counts and detected issues to usage collection ( #64074 , #64459 , #64198 ) Remove cluster autoscaler v1, rename the subcluster label key to ray-subcluster , avoid scaling nodegroups dedicated to the head node, and quiet autoscaling coordinator logs ( #64380 , #64003 , #63918 , #63534 ) Delay the "cluster resources not enough" warning until an operator is persistently starved, and lower the high-memory warning threshold ( #63969 , #64124 ) Remove ExecutionPlan , _num_outputs , batch_format on AllToAllOperators , and InheritBatchFormatRule ; use input_dependencies in logical operators ( #63662 , #64167 , #64152 , #64149 , #64148 ) Deprecate low-level scheduling APIs ( DataContext.scheduling_strategy , actor_locality_enabled , exclude_resources , local:// ) ahead of the actor-only rearchitecture ( #64632 ) Migrate the Daft dependency from getdaft to daft ( #64240 ) Migrate apply_chat_template / tokenize / detokenize callers to *_stage form, and remove PrepareImageStage while deprecating the image row column ( #63590 , #63570 ) 🔨 Fixes Fix TensorDtype.__from_arrow__ crash on empty tensor columns ( #64767 ) Fix Arrow-backed to_pandas regressions with an opt-out flag and int/float block overflow handling ( #64768 ) Fix read-only hash array crash in hash partition ( #64552 , #64559 ) Fix streaming-generator hang on a plasma-resident return object ( #64386 ) Fix Parquet write overflow for string/binary columns larger than 2 GiB ( #64212 ) Fix Parquet sampling hang on a permanent OSError ( #64342 ) Fix the BigQuery datasink to handle empty blocks ( #60797 ) Make PandasBlock.size_bytes deterministic ( #64393 ) Propagate branch in _append_and_commit() for the Iceberg overwrite save mode ( #63922 ) Propagate isolate_read_workers to DatasourceV2 ( #65191 , #65207 ) Fix cross-region S3 bucket access in the obstore download path ( #63890 ) Restore safe_round in the ExecutionResources hot path ( #64296 ) Log both the wrapped and wrapper exception in iterate_with_retry ( #64639 ) Add tf-keras to the text_embedding pip packages ( #64889 , #64968 ) Fix build_processor failing with vLLM >= 0.19, correct its config type hint, and add request_timeout_s to ServeDeploymentProcessor to prevent indefinite hangs ( #64337 , #64098 , #64496 ) Fix type hints in throughput_solver.py ( #64289 ) 📖 Documentation Add a guide on avoiding out-of-memory errors and improve the existing OOM docs ( #64046 , #64297 ) Document the best practice for isolating Ray Data workers from training nodes ( #64588 ) Document running two datasets in one cluster ( #64200 ) Move the read_zarr guide back into its docstring ( #64409 ) Remove Ray Data and DAG docstring ignores from pydoclint ( #63634 ) Ray Serve 🎉 New Features Distribute HAProxy through the ray-haproxy PyPI package, make it the default binary, and remove the from-source build ( #64141 , #64163 , #64164 ) Add gRPC support to the HAProxy direct ingress, including streaming, unary-unary metrics, and custom request IDs ( #63735 , #64310 , #64166 , #64112 ) Emit request and system metrics from HAProxy ( #64329 , #64255 ) Support root_path in the HAProxy ingress ( #64295 ) Add a per-replica metadata hook exposed on ReplicaSelection ( #63948 ) 💫 Enhancements HAProxy stability: graceful drain waits for in-flight and queued requests, replicas quiesce before reporting shutdown complete, dispatches cancelled by replica gRPC stop are retried, reload takeover is verified by pid, and close-spread-time is set in the template ( #63886 , #63920 , #63995 , #64022 , #64063 , #63996 ) Make direct-ingress port reconcile incremental and skip it entirely when ingress membership is unchanged ( #64508 , #64676 ) Controller performance: O(1) version-filtered replica counts, in-place health-check reconcile in the deployment-state loop, and a fast-path orphaned-actor check ( #64699 , #64507 , #64511 ) Freeze GC after replica initialization by default ( #64742 ) Eager-load proxy imports ( #64113 ) Pin the replica actor to bundle 0 of its placement group and track actor fallback_strategy on DeploymentSchedulingInfo ( #62693 , #64346 ) Deprecate HTTPOptions.location in favor of proxy_location , raise on non-zero HTTPOptions.num_cpus , and warn on other deprecated HTTP options ( #64479 , #64418 , #63604 ) Raise a clear error when a custom request router or serve.multiplexed is used on an ingress deployment under direct ingress ( #64211 , #64045 ) Recover ingress request router pin-misses via the fallback proxy instead of returning 503 ( #64218 ) Add RAY_SERVE_CONTROLLER_METRICS_INCLUDE_HIGH_CARDINALITY_TAGS to control controller metric tags ( #63642 ) Default RAY_SERVE_PORT_QUARANTINE_S to hard-stop-after plus a margin ( #64021 ) Defer the "application ready" log until proxies are ready ( #64010 ) Dedup ingress replica and proxy metrics code, and migrate parse_uri from _private to _common ( #64041 , #64371 ) Enforce mypy and pyrefly type checking on clean Serve files ( #64662 ) Fail loudly when subclassing @serve.ingress with a sync __init__ ( #63413 ) Supply a safe default request id header value in direct ingress ( #64118 ) 🔨 Fixes Fix serve.ingress(FastAPI()) pickling and include_router under FastAPI >= 0.137 ( #64814 , #64531 ) Fix an idle worker leak that caused OOM on the Ray head node ( #63597 ) Fix serve.shutdown() skipping live shutdown when the cached controller client is stale after a driver reconnect ( #64660 ) Preserve the deployment actor class across a config re-apply ( #64411 ) Fix rank corruption on controller recovery after a lightweight reconfigure ( #64636 ) Fix route=None crash in direct-ingress metrics reporting ( #64645 ) Fix the direct-ingress backpressure counter leak on cancel-while-queued and clean up the leaked receive task on request timeout ( #64348 , #64123 ) Fix HAProxy healthz falling through to 404 when there are no backends ( #64582 ) Guard request router backoff against OverflowError ( #64539 ) Fix a slot leak race in SingletonThreadRouter.choose_replica ( #63649 ) Fix a type regression in RAY_SERVE_HAPROXY_CLOSE_SPREAD_TIME_S ( #64752 ) Fix Serve tests broken by protobuf 7 ( #64362 ) 📖 Documentation Fix the RequestRouterConfig stats docstring to reference record_routing_stats ( #64086 ) Ray Train 🎉 New Features Add preemption handling: a preemption watcher for node-drain observability, with the signal fanned out to workers ( #63807 , #64099 ) 💫 Enhancements Make Train v1 and v2 use the autoscaling coordinator ( #64824 , #64936 ) Restrict ray.train.report(checkpoint) to in-band checkpoints only ( #63645 ) Make Result.from_path read-only ( #64340 ) Add compute-time metrics to tune_torch_benchmark.py ( #64274 ) Update TensorFlow/Keras examples and tests for Keras 3 compatibility ( #64276 ) 🔨 Fixes Fix CometLoggerCallback mutating the caller's result dict via result.pop() ( #64570 ) Fix the WorkerMetricsCallback method name to match the WorkerCallback interface ( #64568 ) 📖 Documentation Document the unsafe deserialization risk in TorchCheckpoint.get_model() and FrameworkCheckpoint.get_preprocessor() ( #64586 ) Fix docstring style and enable --check-style-mismatch=True in pydoclint ( #63988 ) Enumerate the accepted resource keys for ScalingConfig ( #63449 ) Ray Tune 💫 Enhancements Make BayesOptSearch float-hash precision configurable ( #63914 ) Require optuna>=3.0.0 in OptunaSearch ( #64242 ) 🔨 Fixes Respect max_concurrent_trials for custom searchers ( #63770 ) 📖 Documentation Warn when BayesOptSearch stops early on convergence, and document patience / skip_duplicate ( #64288 ) Restructure the Tune example pages and add PyTorch Lightning examples to the examples README ( #63460 , #63438 ) Document nested-metric keys in ResultGrid.get_best_result and add a conditional search-space example to tune.sample_from ( #63445 , #63443 ) Ray LLM 🎉 New Features Add KV-cache-aware request routing: KVAwareRouter / KVRouterActor interfaces, replica tracking, an event plane, pre-routing tokenization, token-level request lifecycle tracking, and prefill/decode token-load-aware routing ( #64084 , #64085 , #64097 , #64224 , #64327 , #64400 ) Switch the Ray Serve SGLang engine to the sglang RayEngine , reach control-plane parity, and support Ray Serve direct streaming by serving SGLang's native OpenAI app ( #62888 , #63021 , #64611 ) Add the MoRIIO KV-connector backend for prefill/decode disaggregation, with cross-node worker IP advertisement via a vLLM plugin ( #63951 , #64067 ) Enable the /classify and /pooling endpoints in direct-streaming mode ( #64494 ) Add a Ray Serve LLM SGLang metrics dashboard ( #64797 , #65083 ) 💫 Enhancements Upgrade vLLM to 0.25.1 ( #64082 , #64483 , #64697 ) Delegate prefill/decode orchestration to the KV-connector backend and avoid double prompt tokenization in disaggregation ( #63950 , #64049 ) Make request.request_id authoritative for the engine and stop clobbering an explicitly set request id with the Serve id ( #63949 , #64044 ) Parse the direct-streaming routing payload for body-aware routers, and add direct streaming session-affinity tests ( #64328 , #63927 , #64266 ) Use asyncio.create_task instead of ensure_future in KVRouterActor ( #64546 ) Add a cluster filter to the Serve LLM Grafana dashboard ( #64596 ) Validate accelerator_type for CPU vLLM engine configs ( #64235 ) Use model_id rather than the remote URI as the cache identifier in VLLMEngineConfig ( #64110 ) Remove the Serve LLM config generator ( #64075 ) 🔨 Fixes Fix malformed PromQL in the Serve LLM Grafana dashboard ( #63893 ) 📖 Documentation Add a custom vLLM model guide and update the vLLM embedding model example ( #64594 , #64504 ) Improve the Serve LLM docs landing pages, add a config reference, and document direct streaming and the ingress request router ( #64076 , #63860 ) Fix 404s in the Serve LLM examples sidebar links ( #64209 ) Ray RLlib 🎉 New Features Add a pull-based EnvRunnerStateServer for async weight sync ( #63849 ) Expose the count of EnvRunner s dropped on timeout_seconds calls ( #63493 ) 💫 Enhancements Upgrade the example vision stack to MobileNet v3 and remove MobileNet v2 torchhub usage ( #64031 , #64028 ) 🔨 Fixes Narrow from_checkpoint expectations to avoid silent failures ( #63614 ) Require MultiRLModuleSpec.rl_module_specs to be a dict ( #64785 ) Fix the perpetual pending state in offline evaluation during Tune ( #64453 ) Fix RLlib with Tune by reverting the AlgorithmConfig.to_dict() change for the new API stack ( #63695 , #64501 ) Fix ONNX export under torch 2.9 by bumping onnxscript and migrating to the dynamo exporter ( #64410 , #64033 ) Apply observation filters in compute_single_action() on the old API stack ( #64088 ) 📖 Documentation Revamp the RL modules docs and speed up the docs tests ( #64505 ) Fix the outdated parametric-action-spaces link in rllib-algorithms ( #63435 ) Ray Core 🎉 New Features Add an embedded RocksDB storage backend for GCS fault tolerance (REP-64), selected via RAY_gcs_storage=rocksdb and RAY_gcs_storage_path ( #63657 ) Add a public API for topology-aware scheduling via topology_strategy ( #63479 , #63740 ) Implement backpressure for async streaming generators, add actor-level backpressure config, and add _num_objects_per_yield ( #64383 , #63310 , #63943 ) Implement RayTaskEventRecorder as the first step of moving task events out of GCS ( #64168 ) Drain the node on SIGTERM before shutting down ray start --block ( #64454 ) Add tpu.dispatch syntax sugar and a JAX profiler for TPU ( #64493 , #62371 ) Publicize __ray_call__ as a DeveloperAPI for running closures on actors ( #64367 ) Add the NVIDIA RTX PRO 6000 accelerator type and update the AMD accelerator manager ( #64428 , #62477 ) RDT: enable the LIBFABRIC backend for NIXL and upgrade NIXL to v1.2.0 ( #62339 , #63980 ) Add an owner-driven FreeLocalObjects RPC and remove the original FreeObjects RPC and ObjectEviction pubsub ( #63218 , #63181 ) 💫 Enhancements Actor scalability: prefill the placement-group bundle cache at creation to remove an actor submission bottleneck, and construct gRPC metric objects once per process instead of per request ( #64649 , #64732 ) Bound the GCS worker table by evicting old dead-worker entries, prioritized by exit type ( #64612 , #64729 ) Batch object-manager Pull RPCs by destination node and read spilled object chunks in bulk instead of byte-by-byte ( #64225 , #63830 ) Post all object pubsub publishes to the IO thread to relieve lock contention ( #63983 ) Autoscaler: improve v2 scheduler performance for large clusters, handle deleted node types in the metrics reporter, and support env-var configuration for log rotation ( #64175 , #64184 , #63955 ) Integrate IOContextMonitor into GCS, update its metrics, and exclude ray_syncer_io_context and task_io_context from health checks while raising the probe deadline to 30s ( #63930 , #63975 , #64522 , #64421 ) Continue the ClockInterface migration in the core worker and dependency-inject PeriodicalRunnerInterface ( #63956 , #63994 , #64061 , #64029 ) Enable process_group_cleanup_enabled by default and fix graceful-shutdown cleanup ( #64407 ) Make owned object metrics lazy, compute per-component memory usage in MiB, scope prom_metrics_service_discovery.json to the session dir, and set the OpenTelemetry resource identity ( #64093 , #63932 , #63850 , #63921 ) Re-emit per-state task, actor, and placement group gauges every metrics tick and retain gauge observations for a TTL, so live series survive a missed scrape ( #64524 , #64633 ) Add the GPU UUID to the labels of GPU metrics ( #65113 , #65134 ) Log a hint to the driver when system overhead exceeds reserved system memory ( #64270 ) Validate explicit zero resource isolation overrides and move memory monitors into common/monitor ( #63864 , #64491 ) Prioritize tasks over actors when killing under memory pressure ( #64170 ) Bail on AdjustWorkerOomScore read failure and clamp the score to [-1000, 1000] ( #62713 ) Cap the runtime_env parse_uri package name length to avoid ENAMETOOLONG ( #64339 ) Scope the raylet's forced GC to backstop actor handle cycle deadlocks ( #63939 ) Support joining fibers before exiting the fiber runner thread ( #64405 ) Internal cleanups: remove callbacks from TaskToExecute , split task/resource preparation utils out of _raylet.pyx , consolidate the GCS dedicated io_context policy, and rename OnDemandBroadcasting ( #64182 , #63088 , #63931 , #63855 ) 🔨 Fixes Fix a Python 3.14 async-actor memory leak by re-anchoring the stack ( #65177 ) Fix a deadlock between metric registration and collect() in OpenTelemetryMetricRecorder ( #64946 , #65094 ) Fix multi-threaded actor graceful shutdown ( #64019 ) Streaming generator fixes: remove ref stream callbacks that caused a reconstruction hang, fail fast on a replay object-count mismatch, remove the implicit __dealloc__() , and honor timeout_s in the end-of-stream ray.get ( #64581 , #64394 , #64333 , #64014 ) Fix pipelining of actor argument fetches while an actor task is running ( #63079 ) Fix a dangling reference in GetSchedulingClassDescriptor ( #64707 ) Fix a pgid cleanup leak in the job supervisor ( #64746 ) Fix the raylet drain path reading stale object store memory usage ( #64415 ) Avoid a GCS crash on Redis connection loss in RedisResponseFn ( #64204 ) Fix the actor working_dir being overridden by the job-level py_driver_sys_path ( #63756 ) Fix RAY_CHECK failures caused by double ray.cancel() and keyboard interrupts ( #63663 ) Fix event_logger and export_event_logger to flush all handlers safely ( #63947 ) Default TaskProfileEvent.extra_data_ to {} to avoid a JSON decoding error in the State API, and normalize the cmdline field in the StatsPayload schema ( #64589 , #64286 ) Autoscaler v2: fix ALLOCATION_TIMEOUT worker replacement ordering and the RAY_STOP_REQUESTED → RAY_RUNNING fallback when drain has succeeded ( #63815 , #63424 ) Jobs: replace an assert with RuntimeError in put_status and avoid mutating runtime_env during submission ( #64569 , #63990 ) Improve the error message when RAY_ADDRESS is set to an HTTP address ( #64180 ) Define WIN32_LEAN_AND_MEAN globally for Windows builds ( #64361 ) Wait for subprocesses to terminate in ray.shutdown() ( #63655 ) Fix the per-process GPU memory utilization dashboard stat ( #61412 ) 📖 Documentation Update the OOM documentation to use resource isolation mode ( #63652 ) Clarify the KubeRay autoscaler restartPolicy requirement ( #64855 ) Document the name parameter for task.options() ( #63450 ) Enforce Doxygen documentation style for coding agents and remove Ray Core docstring ignores from pydoclint ( #64188 , #63639 ) Dashboard 🎉 New Features Implement the frontend UI for Platform Events ( #63332 ) 💫 Enhancements Link workers to actor detail pages and handle nil IDs ( #63795 ) Support a Grafana Cloud metrics fallback ( #64138 ) Format RuntimeEnvState.creation_time_ms as a duration ( #64207 ) Diagnose dashboard-agent event-loop stalls ( #64436 ) Log errors when job submission fails in JobHead and stop configuring the root logger on module import ( #64443 , #64463 ) 🔨 Fixes Fix local file inclusion in the /logs API endpoint by rejecting absolute paths and .. components in LogAgentV1Grpc.ListLogs() ( #64701 ) Fix TPU metrics ( #63998 ) Fix task grouping by custom task name in the Dashboard API ( #63820 ) Fix an "automaticaly" typo in the Grafana dashboard descriptions ( #64614 ) Ray Wheels and Images Build and publish Python 3.14 wheels and images, add py3.14 TPU base deps, refresh the py3.14 locks, and add a py3.14 smoke release test ( #64754 , #64750 , #64857 , #64938 ). Build and publish cu130 variants for the ray and ray-extra images ( #63972 , #63801 ). Add a py3.13 ray-ml image with torchft-nightly ( #63587 ). Add ray-haproxy to the ray[serve] extra and base requirements, and bump it to 2.8.25 for CVE-2026-55203 /55204 ( #64141 , #64430 ). Unify requirements_compiled across Python versions and upgrade the HuggingFace stack to datasets 4.x ( #64257 , #64054 ). Fix the vendored RocksDB build: use Make instead of Ninja to repair aarch64 core builds, drop -Wl,-pie from the TSAN link flags, and silence TSAN false-positive races ( #64748 , #64917 , #64937 , #64759 ). Documentation Added AGENTS.md with the AI-assisted contribution policy, routed .claude/CLAUDE.md to it, and added a documentation style and grammar guide ( #64419 , #64518 ). Converted the ray-contribute developer guides from RST to MyST, soft-wrapped prose across doc/source , and added an rst-to-myst conversion skill ( #64100 , #64111 , #64115 , #64136 , #64259 , #64279 , #64135 ). Added a unified APIs navigation section, generated API stubs without a full doc build, and greened the API-doc consistency check with reverse/dedup policies and a split whitelist ( #64387 , #64423 , #64420 , #64783 , #64786 , #64808 , #64818 ). Added curated page descriptions for nav pages (SEO and llms.txt ) and a custom 404 page with absolute URLs ( #64330 , #64603 ). Built Read the Docs incrementally from the doc build cache, repaired cache production, and skipped RtD PR builds for code-only PRs ( #64277 , #64414 , #64482 , #64599 , #64547 , #64480 ). Hardened doc template fetches: pinned to specific build ids, guarded zip extraction against path traversal, failed the build loudly on fetch failure, and removed outdated templates now synced at build time ( #64637 , #64669 , #64629 , #64635 , #63971 ). Upgraded Sphinx to 8.2.3 on Python 3.11, decoupled Sphinx from the test/CI dependency stack, and fixed moved intersphinx inventory URLs ( #64070 , #63962 , #64516 ). KubeRay: added an IPPR user guide, documented RayCronJob timezone support and autoscaler command overwrite, updated the RayJob agent sandbox docs, and refreshed the Serve High Throughput guide for Ray 2.56 ( #63212 , #64444 , #63627 , #64460 , #64144 ). Added notebook tests for the lightweight Ray Core examples and recorded the Train example notebooks that are intentionally untested ( #64313 , #64318 ). Refreshed the ecosystem page, added a Ray + Agent Sandbox example, and renamed the AI chat widget to "Ray Docs" ( #64156 , #64047 , #64351 ). Thanks Many thanks to all those who contributed to this release! @justinvyu , @neuyilan , @bveeramani , @abhishekverma-ray , @dstrodtman , @yinli-systems , @Truc54 , @tanmayrauth , @ArchishmanSengupta , @yuhuan130 , @skpark-rh , @shaun0927 , @ps2181 , @edoakes , @htvien , @xinyuangui2 , @owenowenisme , @ronny-anyscale , @nadongjun , @LeMinhNhat2901 , @kevin85421 , @akyang-anyscale , @jeffreywang88 , @sai-miduthuri , @robertnishihara , @andrewsykim , @iamjustinhsu , @leewyang , @marwan116 , @MortalHappiness , @fscnick , @ayushk7102 , @ArturNiederfahrenhorst , @rayhhome , @nh-atuan , @kimngoc280105 , @TimothySeah , @zyxue , @antoine-galataud , @omkar-334 , @alimaazamat , @jhasm , @ShockYoungCHN , @Kunchd , @richardliaw , @xyuzh , @linh285 , @shorbaji , @ans9868 , @LuciferYang , @joaquinhuigomez , @liujp , @ShuChenLin , @johntaylor-cell , @vickytsang , @abrarsheikh , @saivedant169 , @rmhowe425 , @kyuds , @kunling-anyscale , @fuxi611 , @karticam , @enginarslan1 , @spencer-p , @MengjinYan , @AyushKashyapII , @dragongu , @Sparks0219 , @rueian , @praneethkaturi , @dinhxuanvu , @lonexreb , @AarryaSaraf , @Junyi-Wang-6 , @saschwartz , @odncode , @vicentefb , @JasonLi1909 , @coqian , @kouroshHakha , @daiping8 , @pseudo-rnd-thoughts , @wanadzhar913 , @HungHiHung10 , @goutamvenkat-anyscale , @LeThienTrong , @alexandrplashchinsky , @raulchen , @ryankert01 , @jiangxt2 , @HirokiNariyoshi , @Yicheng-Lu-llll , @martinlhw , @richabanker , @aaronscalene , @SohamRajpure , @tvaucher , @Jade07-1 , @eicherseiji , @dayshah , @Kropiunig , @machichima , @harshit-anyscale , @elliot-barn , @liulehui , @OneSizeFitsQuorum , @alexeykudinkin , @sampan-s-nayak , @prince8273 , @jpatra72

Highlights Ray Data Stability: In this Ray release, we've added a variety of stability improvements, including running multiple datasets in a cluster, adding automatic batch size selection to CPU-based map-batches, and default logical memory configuration to prevent OOMs. We've also tightened iter_batches stability by reducing hidden buffering and shutting down the executor when consumers exit early ( #63660 , #63682 , #62949 ). This reduces object-store spilling for common training workloads Ray Serve: We re-architected Ray Serve LLM by decoupling request handling from token streaming response path ( #62667 , #62680 , #62668 , #62669 , #63167 ), resulting in significant LLM serving performance improvements. We've also introduced new routing policies such as session-sticky routing via consistent hashing with ConsistentHashRouter ( #62905 , #63096 , #62906 ) and CapacityQueueRouter ( #62323 ) which is beneficial for supply-constrained workloads. Ray Core: We've added GPU-domain-aware placement groups using label locality ( #61442 , #61614 , #62487 , #62533 ). This enables placement groups to pack bundles onto nodes that share a ray.io/gpu-domain label instead of only packing at the single-node level. We've also added initial Kubernetes in-place pod resizing support for Autoscaler v2 ( #55961 , #62369 , #62215 ), enabling Ray clusters to resize CPU and memory on existing worker pods before scaling out new pods. Ray Data 🎉 New Features Support multiple datasets per cluster via subcluster labels and resource partitioning ( #63331 , #63375 , #63982 ) Add Dataset.mix() public API and MixOperator for weighted dataset mixing ( #63168 , #62450 ) New DataSourceV2 framework: ParquetDatasourceV2 , chunked reader, predicate splitting, listing/scanner infra ( #63113 , #63454 , #63163 , #62975 , #63027 , #62182 ) Add batch_size='auto' to map_batches to derive batch row count from target row batch size ( #62648 ) Implement distributed upsert for Iceberg using task-based merge algorithm, preventing performance bottleneck on driver ( #63482 ) Add include_row_hash to read_parquet ( #61408 ) Add JAX data iterator ( #61630 ) Expose flag to run read tasks on isolated worker processes via isolate_read_workers ( #63490 ) Expose flag to set default logical memory for map operators via default_map_logical_memory_enabled ( #63814 ) Support predicate pushdown for Lance format ( #61400 ) Support per-partition start_offset and end_offset for read_kafka ( #61620 ) Add obstore async download backend for download operator ( #61735 ) Support UDF retries on transient exceptions ( #63023 ) 💫 Enhancements Fix iter_batches spilling by replacing make_async_gen with iter_threaded and reducing buffered batches ( #63660 , #63682 ) Gate restore_original_order in iter_batches behind preserve_order ( #63792 ) Convert drop_columns to a Project logical operator when input schema is known ( #63813 ) Make ConcatAggregation and TurbopufferDatasink use polars for sorting ( #61904 ) Boost and vectorize hash_partition with sort_indices , zero-copy slices, and pandas ( #63498 , #62757 , #63152 , #62587 ) Enable GPU_SHUFFLE in grouped_data.py ( #62410 ) Eager StarExpr expansion, schema inference for non-black-box UDFs, and Expressions struct support ( #63776 , #63387 , #62560 ) Make logging configurable via RAY_DATA_LOG_LEVEL and log RAY_DATA env vars at execution start ( #63487 , #63380 ) Display and track logical memory in progress bar ( #63379 ) Honor compute= in filter(expr=...) and deprecate concurrency= ( #63576 ) Enable filter pushdown through StreamingRepartition and read stage column-rename removal ( #62347 , #63384 , #63582 ) Cache deserialized Arrow schemas in BlockMetadataWithSchema ( #63462 ) Track scheduling-loop step duration (p50/p90/max), peak USS/object-store memory, and task block locality ( #63586 , #63345 , #63489 , #63418 , #62249 ) Replace TaskDurationStats and Timer with DistributionTracker ( #63488 , #63530 , #63825 ) Introduce BlockEntry on RefBundle in place of (ref, metadata) tuples ( #63654 ) Pre-resolve filesystem in threaded download to avoid IMDS herd ( #62898 ) Convert logical operators to frozen dataclasses and consolidate operator base/repr ( #62593 , #62568 , #62400 , #63137 , #63140 , #63108 ) Non-blocking default autoscaling coordinator and resource-aware auto-downscaling ( #62725 , #62574 ) Release pinned blocks after dataset execution and shut down executor on early DataIterator exit ( #62456 , #62949 ) Optimize local shuffle with incremental index and configurable compaction threshold ( #62539 ) Speed up checkpoint filter and reduce memory usage ( #60294 ) Preserve Arrow types through pandas roundtrip and reorder block columns by name before schema ops ( #63017 , #63582 ) Block pickle object columns when reading untrusted Parquet and gate unsafe WebDataset deserialization ( #63470 , #63469 ) Move backpressure escape hatch across all policies ( #63539 ) Update pandas , modin , and pyarrow minimum versions ( #62899 ) Add utilization monitoring and correct logical resource usage for ActorPool ( #61987 , #61528 ) Deprecate ConcurrencyCapBackpressurePolicy , DataIterator.to_torch , and pandas UDF batches ( #63392 , #62540 , #61733 ) Rank actors per node in a heap and avoid re-exporting actor class via .options ( #62309 , #62722 ) read_delta reads from preconfigured pyarrow dataset ( #61721 ) Include column name and target type in ArrowConversionError ; reduce arrow conversion warning verbosity ( #62407 , #61486 , #62521 ) Show external consumer bytes in verbose operator progress log ( #63728 ) Disable DataSourceV2 by default after earlier enabling ( #63674 , #63326 ) 🔨 Fixes Rename subcluster label key from __subcluster__ to ray-subcluster ( #63982 ) Fix get_or_create_stats_actor crash in Ray Client mode ( #63402 ) Fix datasource pushdown crashes for generic UDFExpr filter predicates ( #63781 ) Fix hash-shuffle aggregator memory estimation: metadata propagation, node-size clamp, column pruning ( #63809 ) Fix CheckpointConfig FileNotFoundError on Azure Blob Storage ( #63606 ) Fix silent credential drop for fsspec-S3 in download expression ( #62897 ) Fix missing f-string prefix in _concatenate_extension_column ( #62939 ) Fix HashAggregate duplicate group rows for AggregateFnV2 ( #63066 ) Fix JSONL read retry with advanced file cursor ( #63233 ) Fix read_parquet ArrowNotImplementedError for nested column types exceeding ~2GB row group ( #61824 ) Fix read_parquet nested-type fallback and parquet scanner memory accumulation ( #63175 , #62745 ) Fix memory leak in DataIterator.to_torch() by switching to PyArrow ( #60966 ) Fix ZipOperator freeing shared blocks via _split_at_indices ( #62665 ) Fix concurrent writes race condition in write_parquet ( #62377 ) Fix GPU shuffle output ordering when using ShuffleStrategy.GPU_SHUFFLE ( #62351 ) Fix incorrect DatasetStat uuid propagation ( #62255 ) Fix none issue when DATA_ENABLE_OP_RESOURCE_RESERVATION=False ( #61718 ) Fix filesystem compatibility check for fsspec-wrapped PyFileSystem ( #61850 ) Forward try_create_dir to pyarrow.dataset.write_dataset ( #58302 ) Fix autoscaler bug blocking timely release of leased resources ( #62592 ) Ensure consistent nan_is_null /nans-as-nulls semantics in encoder ( #62623 , #62618 ) Skip unconditional null strip in find_partition_index ( #62594 ) V1 _split_predicate_by_columns correctness fix ( #63176 ) Avoid importing cudf in _is_cudf_dataframe when cudf not loaded ( #62302 ) Revert raw-modulo hash partition fast path ( #63097 ) Remove tfx-bsl support from read_tfrecords ( #63245 ) 📖 Documentation Document isolate_read_workers for read_parquet ( #63816 ) Remove docs recommending increased object store memory proportion ( #63389 ) Update docs minimum version for build_processor and "auto" batch size ( #61757 , #62790 ) Remove outdated limitation of DefaultClusterAutoscalerV2 and stale object-store-memory warnings ( #62385 , #62387 ) Ray Serve 🎉 New Features: Add custom ingress request router app interfaces and HAProxy ingress dispatch path ( #62680 , #62668 , #62669 , #62667 ) Expose choose_replica / dispatch on deployment handles and AsyncioRouter with replica-side slot reservation ( #63255 , #63254 , #63252 ) Introduce experimental round robin router and ConsistentHashRouter for session-sticky routing ( #63238 , #62906 , #63096 , #62905 ) Central capacity queue for token-based request routing via CapacityQueueRouter ( #62323 ) Add experimental ray-haproxy support behind RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY ( #62589 ) Add deployment actor context API and broadcast API for deployment handles ( #62532 , #61472 ) Add ControllerOptions for configurable controller runtime_env ( #63352 ) Make rolling update percentage configurable ( #62160 ) Support per-request timeout and disconnect in HTTP proxy path ( #62867 ) 💫 Enhancements: HAProxy stability improvements: wait for old workers before drain, redirect stdout/stderr, redispatch+retry-on, coalesce broadcasts, quarantine released ports ( #63620 , #63621 , #63622 , #63623 , #63628 ) Bind direct ingress ports to 0.0.0.0 for cross-node HAProxy routing ( #62515 ) HAProxy ingress request router metrics, enable splice by default, TCP_NODELAY default 1, optional retry knobs, RAY_SERVE_HAPROXY_STATS_PORT ( #63356 , #63531 , #63353 , #63415 , #62979 ) Resolve bundled ray-haproxy binary before RAY_SERVE_HAPROXY_BINARY_PATH ; HAProxy abspath env var ( #63829 , #62610 ) Replace socat subprocess with Python socket for HAProxy admin communication; bump HAProxy to avoid CVE-2025-11230 ( #61897 , #62585 ) Expose controller health metrics via /api/serve/applications/ API; add max_replicas_per_node to response ( #63556 , #63234 ) Run health check on user execution path to detect request-serving stalls ( #61621 ) Mark widely-used APIs as stable ( #62932 ) Retain recently-stopped replica logs in the dashboard ( #63678 ) Add observability logs for pack scheduling decisions ( #63603 ) Gate ingress request router body forwarding behind escape hatch ( #63183 ) Avoid rolling replicas for no-op config overrides ( #63034 ) Gate replica/deployment creation during shutdown ( #62761 ) Defer PG creation for TPU Serve deployments to accelerator backend ( #62941 ) Expose DeploymentStateManager APIs for controller access ( #62950 ) Add tracing support for Windows and gRPC tracing improvements ( #62821 , #63833 ) Split node vs requested resources in deployment scheduler ( #62778 ) Defer DEPLOYMENT_TARGETS broadcast while replicas are RECOVERING ( #62751 ) Evict per-deployment LongPollHost state on deployment delete; enable logs when client stops its event loop ( #62820 , #63028 ) Add metrics: max replica processing latency, objref resolution latency, serve_autoscaling_target_ongoing_requests ( #62381 , #62355 , #62421 ) Filter stale bootstrap observations from serve_long_poll_latency_ms ( #62868 ) Retry build_serve_application task on failure ( #62987 ) Scale down non-matching primary-label replicas first ( #61488 ) Refactor internal autoscaling policy state extraction into a single helper ( #62452 ) Catalog Ray Serve env vars ( #62006 ) Remove or raise clear error for deprecated deployment items; remove deprecated DeploymentMode ( #63548 , #63510 ) 🔨 Fixes: Fix orphaned actors on controller crash during shutdown; drop and replace replicas surviving a controller crash without rank assignment ( #62823 , #63139 ) Fix deployment actors creating 15K OS threads for sync actor classes ( #62661 ) Fix gang scheduling PG leak when deployment actors are starting ( #62469 ) Fix app-level autoscaling policy state cross-deployment contamination and state loss for skipped deployments ( #62484 ) Fix Serve autoscaling delay to use wall-clock time ( #62144 ) Fix race condition in multiplex LRU cache update using move_to_end() ( #62548 ) Normalize multiplexed model ID header to support proxy-transformed names ( #61869 ) Fix AttributeError when request_router is None in update_deployment_config ( #63180 ) Fix potential UnboundLocalError in ActorReplicaWrapper.check_stopped() ( #63339 ) Fail loud when ingress request router dispatch fails ( #63215 ) Fix stale _global_client cache across driver sessions ( #62368 ) Fix start_metrics_pusher crash when deployment has record_autoscaling_stats but no autoscaling config ( #62123 ) Fix high-cardinality namespace tag on long poll metrics ( #62386 ) Fix Java long poll timeout serialization ( #61875 ) Avoid destructor error when FastAPI ingress init fails ( #62172 ) Avoid proxy readiness future timeout race ( #62194 ) Avoid self-cause on non-gRPC replica exceptions ( #62412 ) Fix HAProxy startup timeout propagation ( #61752 ) Include ingress_request_router.lua.tmpl in package_data ( #63145 ) Revert support for root_path parameter across uvicorn versions ( #62529 ) 📖 Documentation: Add round robin and consistent hashing router documentation ( #63636 ) Introduce gang scheduling documentation ( #61737 ) Add deployment scope actor docs ( #62735 ) Add Kuberay guide for RayService with HAProxy and High Throughput mode ( #62408 ) Add Ray Serve office hours invite into documentation ( #62176 ) Ray Train 🎉 New Features Add LoggingConfig for configuring the ray.train logger on controller and workers ( #61550 ) Allow DataParallelTrainer 's train_fn to return data ( #62021 ) Add async checkpointing/validation with Torch Lightning ( #62370 ) 💫 Enhancements Report time spent syncing and transferring checkpoints to storage in ray.train.report(checkpoint) ( #62027 ) Block until create_or_update_train_run completes on Train initialization ( #63432 ) Implement DatasetManager ( #63309 ) Forward label_selector to AutoscalingCoordinator ( #63287 ) Add log line before launching training function ( #62911 ) Allow contextlib.redirect_stdout() to bypass print redirect to logs ( #61075 ) Add timeouts to validation functions of ray.train.report ( #62916 ) ray.train.report does not hang across replica group restarts; Ray Train manages replica group restarts ( #62651 , #61475 ) Swallow RayTaskError during BackendSetupCallback shutdown ( #63143 ) Improve JaxTrainer TPU multi-slice fault tolerance and reservation ergonomics ( #62893 ) Export default data execution options ( #62784 ) Consolidate Train run metadata sanitization and improve readability ( #63182 ) Fix PlacementGroupCleaner race condition: drain queue before cleanup on controller death ( #62754 ) Harden against unsafe pickle deserialization ( #62807 ) Raise error when checkpoint is within experiment directory and delete_local_checkpoint_after_upload=True ( #62555 ) Add timeout_s to ray.train.get_all_reported_checkpoints ( #61761 ) Change remaining pytorch_lightning imports ( #61291 ) Make controller resilient to errors in all lifecycle hooks ( #60900 ) Remove Predictor from Train v1 ( #63461 ) 🔨 Fixes Fix missing comma in DataBatchType Union type ( #63872 ) Handle Arrow-backed pandas dtypes in LightGBM examples ( #63427 ) Fix exclude_resources regression for V1 Train + V2 cluster autoscaler ( #62827 ) Add missing %s to logger.debug ( #63039 ) Increase get_actor timeout ( #62516 ) 📖 Documentation Document S3-compatible storage ( #63103 ) Add Azure Files to persistent storage docs ( #63406 ) Uncomment Result.from_path in docs ( #62887 ) Document how to tune async validation ( #62227 ) Document why validation runs need unique names ( #62224 ) Ray Tune 💫 Enhancements Fix Tune search for Python 3.14 ( #63575 ) Modernize AxSearch for Ax Platform 1.0.0+ ( #60522 ) Use built-in inspect for argument capture ( #60049 ) 🔨 Fixes Fix import count in CIFAR PyTorch tutorial ( #62756 ) Ray LLM 🎉 New Features Major Ray Serve LLM performance improvement with direct streaming ( #63167 , #63468 , #63779 ) TPU support: Add topology field to LLMConfig for multi-host TPU support ( #61906 ) Add per-host bundles default and fix fractional TPUs for TPUAccelerator ( #63177 ) Enable Ray Serve LLM session-stickiness routing policy via RAY_SERVE_SESSION_ID_HEADER_KEY ( #63362 ) 💫 Enhancements Upgrade vLLM to 0.22.0 ( #63730 , #63396 , #62970 , #62349 ) Co-locate DP rank 0 worker with advertised master address ( #63803 ) Add pick-only fast path to AsyncioRouter for LLM ingress ( #63517 ) Replace LLM ingress router replica selection with choose_replica ; don't fetch LLMConfig from replicas at startup ( #63280 , #63065 ) Promote max_tasks_in_flight_per_actor to a first-class config field and adjust defaults ( #63214 ) Validate accelerator_type against CPU-only configs; replace GPUType alias with AcceleratorType ( #62139 , #62978 ) Add rate-limiter for per-request traceback spam ( #62440 ) Promote SGLang integration to user guide and move engine to _internal ( #62570 ) Lazy-load batch stage/processor submodules and make boto3/botocore imports lazy ( #62861 , #62383 ) LLM telemetry bugfixes ( #63782 ) 🔨 Fixes Fix flaky GPU-0 worker and NIXL port collisions ( #63810 ) Fix P/D direct streaming OpenAI routing ( #63679 ) Remove guided_decoding , truncate_prompt_tokens , build_llm_processor ( #63569 ) Fix misleading ImportError when vLLM is installed but fails to import ( #63305 ) Fix max_pending_requests default to track vLLM 's GPU-dependent max_num_seqs ( #62918 ) Fix HF config loading for models with custom rope_scaling ( #62464 ) Wait for request router init in LLMRouter constructor ( #63206 ) Materialize chat completion message content in sanitizer ( #63119 ) Fix lora_request not forwarded to vLLM engine + add regression tests ( #62609 ) Fix SGLangEngineProcessor telemetry for trust_remote_code models ( #62102 ) Fix TOKENIZER_ONLY downloads missing chat_template for S3-backed models ( #62121 ) Fix SGLang chat tokenize to respect add_generation_prompt ( #61688 ) Fix bool serialization in benchmark_vllm CLI builder ( #63516 ) 📖 Documentation Document multimodal pixel-budget gotchas and vLLM compatibility ( #63593 ) Add tokenization disaggregation documentation ( #62494 ) Add benchmark docs and refactor into submodules ( #62204 ) Remove VLLM_USE_V1 from docs and examples ( #63001 ) Fix wrong documented default for max_tasks_in_flight_per_actor ( #62917 ) Ray RLlib 🎉 New Features Add custom_resources_per_learner config and custom_resources_for_main_process to AlgorithmConfig ( #63303 , #62475 ) Add Importance Sampling APPO metrics to the torch learner ( #63675 ) 💫 Enhancements Put only one copy of weights into the object store ( #63529 ) Handle the all-evaluation-workers-unhealthy case uniformly across modes ( #63128 ) Stop IMPALA / APPO learner thread gracefully to avoid misleading error messages ( #62763 ) Improve invalid input error messages ( #62324 ) 🔨 Fixes Fix two substantial edge cases in PPO 's value target calculation ( #59958 ) Fix EnvRunner crash loops ( #62884 ) Fix extra model outputs hanging val indexing ( #62960 ) Fix ValueError in MultiAgentEpisode.get_rewards() when an agent is inactive for all requested env steps ( #62907 ) Preserve Torch optimizer param-group scalar types on restore ( #61937 ) Fix wrong assert variable in _update_env_seed_if_necessary ( #61823 ) Maintain value in EMAStat ( #63064 ) 📖 Documentation Clarify extra model output docstrings ( #63524 ) Ray Core 🎉 New Features Add support for Furiosa AI NPU ( #63035 ) and register_collective_backend API for custom collective backends ( #60701 ) In-place pod resizing (IPPR) on Kubernetes 1.35: initial implementation and standalone KubeRay IPPR provider ( #55961 , #62369 , #62215 ) Label locality support: GPU-domain-aware placement groups, autoscaler proto changes, and state API observability ( #61442 , #61614 , #62487 , #62533 ) Publish platform events via Ray Event Recorder and support single-event emission in the Python layer ( #63329 , #60858 ) Autoscaler v2: priority-based worker group selection ( #62997 ) and noDriverTimeoutSeconds for KubeRay cluster termination ( #63465 ) RDT: concurrent one-sided transfers for multiple ObjectRef s in ray.get ( #61773 ), retry support ( #62842 ), and NIXL memory deregistration via deregister_nixl_memory ( #62341 ) Support .tar.gz archives for remote working_dir URIs ( #62813 ) Add IPv6 localhost and all-interfaces support ( #60023 ) 💫 Enhancements Resource isolation: event-based memory monitor, multi-memory-monitor factory, time-based group killing policy, idle-worker prioritization, system/user slice bounds, and OOM policy tuning ( #62060 , #62705 , #62643 , #62378 , #62168 , #63521 , #63324 , #63067 , #62957 ) Compute per-component memory usage in MiB ( #63932 ) and add host vs container memory distinction to memory panels ( #63111 ) Consider cgroup limit when fetching CPU ( #63685 ) and correct worker OOM score adjustment logic ( #62470 ) Replace NodeAffinitySchedulingStrategy with Label Selector API when soft=False ( #54940 ) Improve SlicePlacementGroup lifecycle and support explicit bundle_label_selector for TPUs ( #63171 ); add TPU head resource for Ironwood TPU ( #62786 ), chips_per_vm arg ( #62526 ), and v6e single-host fixes ( #62306 ) Batch placement group bundle removal RPCs ( #63839 ); remove PG resource deduction from GCS in favor of resource broadcast ( #63723 ) Migrate Raylet/GCS timing logic to a shared ClockInterface with a fake clock for testing ( #62562 , #62502 , #62476 ) Refactor asio build targets and add IOContextMonitor ; run GCS health check on io_service ( #63042 , #63166 , #62608 , #62374 ) Autoscaler v2 performance: skip serializations for debug logs ( #63778 ); accept fractional resource values in request_resources ( #63306 ) Reduce traffic: halve task arg pubsub by skipping redundant raylet pull ( #62583 ), avoid extra memcpy when spilling fused objects ( #63653 ), and resolve task dependencies synchronously when objects exist ( #62561 ) Improve inspect_serializability messages and traversal context ( #63501 , #63373 , #63258 ); better worker startup error messages ( #63714 ) Warn when runtime_env package approaches upload size limit ( #63404 ); harden zip extraction path containment ( #63786 , #62813 ) Include owner node ID in OwnerDiedError ( #63727 ); add dependency info to taskspec debug string ( #62316 ) Add unexpected worker failure metric and dashboard panel ( #62297 ); group observability APIs in ray CLI help ( #62748 ) Normalize OTel metric labels before Prometheus export ( #63744 ) and retry/log when Prometheus queries fail ( #63578 ); add GPU usage instance filter ( #62214 ) Move observability and control-plane pubsub to dedicated services and rename InternalPubSub* to ControlPlanePubSub* ( #62806 , #63044 , #62461 ) AMD GPU: replace rocm-smi ctypes binding with amd-smi Python interface ( #62393 ); detect NVIDIA Blackwell consumer GPUs ( #63322 ) Add task retry delay for ACTOR_UNAVAILABLE retries ( #62330 ); improve State API filter key handling ( #63638 ) Patch setproctitle to skip launch services IPC calls ( #63366 ); add timeout for first redis probe ( #63148 ) Clarify head node commands in ray up output ( #63409 ); pass logging_config through Ray Client ray.init ( #62192 ) Print subprocess log tails with exit codes on unexpected exit ( #61905 ); add warning log when GPU profiling command times out ( #63706 ) Add unique suffix to log filenames ( #62365 ); disable profiling endpoints by default ( #62531 ) Remove pydantic v1 support ( #62716 ); update Starlette to v1.0.1 ( #63722 ) Deprecate DAGNode.execute() ( #63716 ); remove experimental _owner support for ray.put ( #63520 ) 🔨 Fixes Fix ray.get hanging forever when an object's owner dies during pull ( #63694 ); resolve ReferenceCounter race on WORKER_REF_REMOVED_CHANNEL ( #60495 ) Fix resource leaks in subprocess management ( #63878 ) and runtime_env cache not detecting changes in -r -referenced requirements files ( #63403 ) Fix replica actor zombie process after GCS restart ( #63764 ); fix actor creation race condition ( #62994 ); fix actor state counter bug ( #63647 ) Fix placement groups with label domain stuck on the infeasible queue ( #62483 ); log status for failed PG PrepareResources / CommitResources ( #62836 ) Fix env var expansion in ray job submit CLI via shlex.join ( #63797 ) and --working-dir for local zip files and http:// URLs ( #62843 ) Surface WebSocket close codes and errors in job log streaming ( #63364 ); fix ray stop failing to terminate dashboard/runtime_env agents on Windows ( #62428 ) Fix ray down not stopping Docker containers on worker nodes for local clusters ( #62169 ); fix delayed/missing worker logs in Jupyter by flushing stdout/stderr ( #63599 ) Fix Python log monitor handling for same-inode truncated files ( #63720 ); avoid os.getcwd() on import by lazily evaluating scratch_dir ( #63040 ) Fix accelerator detection on NVIDIA Blackwell consumer GPUs ( #63322 ); avoid FabricManager stall on NVLink systems in GpuProfilingManager ( #63312 ) Fix POSIX semaphore crash in experimental mutable objects ( #62328 ); fix overflow on exponential backoff multiplication ( #62366 ) Fix OOM kill message wrong threshold with resource isolation ( #62948 ); fix OpenTelemetryMetricRecorder singleton init guard ( #63081 ) Fix MarkFootprintAsBusy clearing saved idle state for unrelated items ( #62588 ); fix HandleIsLocalWorkerDead for drivers ( #62688 ) Fix AttributeError on trace in client mode ( #62955 ); fix IndexError in legacy post-mortem debugging ( #61479 ) Keep strong references to fire-and-forget asyncio tasks ( #63291 ); validate JobConfig code_search_path type ( #62499 ) Fix uv existence check in UVProcessor ( #62818 ); fix invalid default stats factory in ClusterStatus ( #62934 ) Fix autoscaler v2 instance_type_name in autoscaling state ( #62101 ) and stopped-node metric double counting ( #62026 ) Fix ReadOnlyProviderConfigReader max_workers counting bug ( #62819 ); fix circular import in ray_print_logs thread ( #63410 ) Fix wrong container in spill-fusion threshold check ( #63605 ); avoid emitting idle worker failure for unregistered failed workers ( #62789 ) Avoid return in finally block (Python 3.14 SyntaxWarning ) ( #63742 ); fix typos and replace type() checks with isinstance() ( #62154 ) 📖 Documentation Add "bring your own transport" docs page for RDT ( #60308 ); doc changes for label locality support ( #62551 ) Fix misleading docstrings on drain_node APIs ( #62942 ); update outdated description for max_direct_call_object_size ( #63164 ) Dashboard 🎉 New Features Add Platform Events module with K8s event ingestion/caching and frontend UI ( #62314 , #63332 ) Show TPU stats on the Cluster tab ( #63774 ) 💫 Enhancements Add py-spy --idle and --subprocesses flags to profiling endpoints ( #63852 ) Pass Grafana cluster filter to Serve metrics URLs ( #63211 ) Show last data load time ( #63618 ) Add Name column to Jobs view from job_name metadata ( #62257 ) Mask password arguments in get_entrypoint_name() to prevent password exposure ( #61995 ) 🔨 Fixes Fix TPU metrics ( #63998 ) Guard against zero num_cpus in k8s_utils.cpu_percent ( #63729 ) Fix invalid PromQL when global_filters is empty in Grafana dashboard generation ( #63687 ) Fix unexpected log line details pop-up in log viewer UI ( #62637 ) Ray Wheels and Images Bumped the Ray version for the 2.56.0 release. Bumped the minimum Python version in pyproject.toml ( #62569 ). Added TPU release images ( #62113 ) and updated the TPU Docker image base dependencies ( #63006 ). Added a torchft image for Torch trainer tests ( #63361 ) and ran apt-get upgrade for slim base images ( #62666 ). Numerous dependency lockfile and CI image updates (raydepsets migration, depset regeneration across core/ML/RLlib/docs/macOS CI images). Documentation Established doc/redirects/current.yaml as the redirects source of truth with legacy-version 404 redirect coverage ( #63367 , #63880 ). Added an agent context guide for Ray documentation and an ipython3 lexer hook for notebook shell/magic cells ( #63227 , #63515 ). Added Sphinx /llms.txt and /llms-full.txt generation, excluding Jupyter notebooks ( #63130 , #63228 ). Upgraded doc toolchain: pydata-sphinx-theme 0.17.1, myst-nb 1.4.0, added sphinxext-opengraph , unpinned yanked tf-keras ( #63344 , #63360 , #63343 , #63358 ). Banned new .rst files under doc/source/ and added CI to skip RTD builds for PRs that don't touch docs ( #63057 , #63431 ). Added meta descriptions to ray-contribute pages and anonymized personal paths in Tune notebook outputs ( #63832 , #63464 ). Tune: updated deprecated sample_from examples to config-dict style and documented time_attr scheduler values ( #63804 , #32467 ). RLlib: clarified DQN hiddens as dueling-only, removed a broken parametric-actions link, fixed broken doc links ( #43051 , #54671 , #47146 ). Ray Data: added a map_batches shuffle section, streaming generator docs, and fixed a broken README link ( #62576 , #63791 , #63412 ). Ray Train: documented iter_jax_batches for JaxTrainer and updated TPU scaling config docs ( #63294 , #62584 ). Kubernetes/TPU: added a GKE Gateway ingress example, fixed the GKE TPU guide, and replaced deprecated example images ( #63546 , #63209 , #63019 ). Added a RayCronJob quick-start guide and clarified KAI Scheduler RayJob submission modes ( #62151 , #61332 ). Added a Slurm guide for running Ray inside Docker containers ( #63221 ). Documented AutoscalingConfig replica/target fields and corrected max_calls default docs ( #48601 , #63894 ). Dependencies This is the last Ray release to support the dependency versions listed below. For the 2.57.0 release, Ray will raise its minimum required versions for several core dependencies. If your environment pins any of these packages below the new minimums, plan to upgrade before moving to the next Ray release. Dependency Last supported in this release New minimum (next release) numpy < 2.1 >= 2.1 protobuf < 5.26 >= 5.26 pandas < 2.2.3 >= 2.2.3 pyarrow < 18.0.0 >= 18.0.0 pydantic < 2.9 >= 2.9 grpcio < 1.66 >= 1.66 scipy (previously unpinned) >= 1.14.1 Most users on recent releases of these packages are unaffected Thanks Many thanks to all those who contributed to this release! @khluu , @Krishnachaitanyakc , @leewyang , @ssam18 , @christian-pinto , @Hyunoh-Yeo , @hango880623 , @yuanzhuoyang1-bit , @marwan116 , @aaronscalene , @tianyi-ge , @TriNguyen1208 , @andrewsykim , @leonaIee, @OneSizeFitsQuorum , @AksodFlare , @limarkdcunha , @dayshah , @jade710 , @pedrojeronim0 , @dev-miro26 , @DonPalius , @TimothySeah , @abrarsheikh , @nathon-lee , @prince8273 , @Bye-legumes , @rayhhome , @Yunnglin , @spencer-p , @ryanaoleary , @herin049 , @stephanie-wang , @liulehui , @slxswaa1993 , @psaikaushik , @cyhapun , @tdat1465 , @akyang-anyscale , @chenshi5012 , @zzchun , @ryankert01 , @EagleLo , @mzjp2 , @justinvyu , @petern48 , @YuangGao , @sjp611 , @wingkitlee0 , @AndySung320 , @dstrodtman , @Accurio , @JasonLi1909 , @peterjc123 , @eicherseiji , @kyuds , @Chong-Li , @joaquinhuigomez , @IrvinFan , @XuQianJin-Stars , @AJamesPhillips , @harshit-anyscale , @claytonlin1110 , @nhquana2 , @Rruop , @win5923 , @raulchen , @rohankmr414 , @andrew-anyscale , @YoyinZyc , @doanxem99 , @liujp , @dancingactor , @Evelynn-V , @SohamRajpure , @dragongu , @ShockYoungCHN , @ljstrnadiii , @WFY123wfy , @axreldable , @pseudo-rnd-thoughts , @H4ck2 , @mvcb , @xinyuangui2 , @edoakes , @ankushbbbr , @ps2181 , @dominikkawka , @vinhuytran0810-cell , @siyuanfoundation , @MengjinYan , @Chronostasys , @jeffreywang88 , @lalitc375 , @sampan-s-nayak , @ArturNiederfahrenhorst , @srini047 , @ChangyuWang , @adam360x , @Yicheng-Lu-llll , @thakoreh , @Aydin-ab , @manhld0206 , @oab24413gmai , @ayushk7102 , @tycao0338-cpu, @slfan1989 , @myandpr , @rueian , @ans9868 , @Ziy1-Tan , @elliot-barn , @as-jding , @daiping8 , @robertnishihara , @MatthewCWeston , @Cursx , @laysfire , @karticam , @Mr-Neutr0n , @jjyao , @zent1n0 , @aslonnie , @DenBuzz , @michael-pryor , @goanpeca , @nadongjun , @ronny-anyscale , @GoparapukethaN , @werkt , @carolynwang , @kamil-kaczmarek , @madiyar-wayve , @peterxcli , @pqkzzz , @Future-Outlier , @iamjustinhsu , @micah-yong-ai , @wxwmd , @owenowenisme , @sai-miduthuri , @lonexreb , @prassanna-ravishankar , @wanadzhar913 , @kouroshHakha , @tobby168 , @johntaylor-cell , @richabanker , @Kunchd , @vincere-mori , @vaishdho1 , @wenhaozhao011-cmd , @bveeramani , @bittoby, @Phucvt123 , @aschuh-hf , @RudrenduPaul , @xyuzh , @Sparks0219 , @yancanmao , @eureka0928, @yuhuan130 , @goutamvenkat-anyscale , @Zerui18 , @machichima , @Lucas61000 , @weimingdiit , @xi377266, @EmaFerrao , @awen11123 , @Lawson-Darrow , @suppagoddo

Fixes SSH connectivity issue in the ray-llm image ( #62625 / #62718 ). Upgrade apt packages in slim base ( #62666 / #62717 ).

Ray Data 🎉 New Features Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning ( #61220 , #61615 , #61997 ) Support GPU shuffle with rapidsmpf 26.2 ( #61371 , #62062 ) Add Kafka datasink, migrate to confluent-kafka , support datetime offsets ( #60307 , #61284 , #60909 ) Add Turbopuffer datasink ( #58910 ) Add 2-phase commit checkpointing with trie recovery and load method ( #61821 , #60951 ) Queue-based autoscaling policy integrated with task consumers ( #59548 , #60851 ) Enable autoscaling for GPU stages ( #61130 ) Expressions: add random() , uuid() , cast , and map namespace support ( #59656 , #60695 , #59879 ) Add support for Arrow native fixed-shape tensor type ( #56284 ) Support writing tensors to tfrecords ( #60859 ) Add pathlib.Path support to read_* functions ( #61126 ) Add cudf as a batch_format ( #61329 ) Allow ActorPoolStrategy for read_datasource() via compute parameter ( #59633 ) Introduce ExecutionCache for streamlined caching ( #60996 ) Support strict=False mode for StreamingRepartition ( #60295 ) Port changes from lance-ray into Ray Data ( #60497 ) Enable PyArrow compute-to-expression conversion for predicate pushdown ( #61617 ) Add vLLM metrics export and Data LLM Grafana dashboard ( #60385 ) Include logical memory in resource manager scheduling decisions ( #60774 ) Add monotonically increasing ID support ( #59290 ) 💫 Enhancements Performance: cache _map_task args, heap-based actor ranking, actor pool map improvements ( #61996 , #62114 , #61591 ) Optimize concat tables and PyArrow schema hashing ( #61315 , #62108 ) Reduce default DownstreamCapacityBackpressurePolicy threshold to 50% ( #61890 ) Improve reproducibility for random APIs ( #59662 ) Clamp batch size to fall within C++ 32-bit int range ( #62242 ) Account for external consumer object store usage in resource manager budget ( #62117 ) Make get_parquet_dataset configurable in number of fragments to scan ( #61670 ) Consolidate schema inference and make all preprocessors implement SerializablePreprocessorBase ( #61213 , #61341 ) Disable hanging issue detection by default ( #62405 ) Make execution callback dataflow explicit to prevent state leakage ( #61405 ) Log DataContext in JSON format at execution start for traceability ( #61150 , #61428 ) Autoscaler: configurable traceback, Prometheus gauges, relaxed constraints ( #62210 , #62209 , #61917 , #61385 ) Add metrics for task scheduling time, output backpressure, and logical memory ( #61192 , #61007 , #61436 ) Prevent operators from dominating entire shared object store budget ( #61605 ) Eliminate generators to avoid intermediate state pinning ( #60598 ) Default log encoding to UTF-8 on Windows ( #61143 ) Remove legacy BlockList , locality_with_output , old callback API, PyArrow 9.0 checks ( #60575 , #61044 , #62055 , #61483 ) Upgrade to pyiceberg 0.11.0; cap pandas to <3 ( #61062 , #60406 ) Refactor logical operators to frozen dataclasses ( #61059 , #61308 , #61348 , #61349 , #61351 , #61364 , #61481 ) Prevent aggregator head node scheduling ( #61288 ) Add error for local:// paths with a zero-resource head node ( #60709 ) 🔨 Fixes Fix RCE in Arrow extension type deserialization from Parquet ( #62056 ) Fix StreamingSplitDataIterator.schema() ( #62057 ) Fix ParquetDatasource handling of FileSystemFactory.inspect ( #62065 ) Fix read_parquet file-extension filtering for versioned object-store URIs ( #61376 ) Fix wide_schema_pipeline_tensors cloudpickle deserialization ( #62149 ) Fix OpBufferQueue race condition ( #60828 ) Fix scheduling metrics computation ( #62031 ) Fix OneHotEncoder max_categories to use global top-k instead of per-partition ( #60790 ) Fix ReservationOpResourceAllocator resource borrowing for ActorPoolMapOperator ( #60882 ) Fix DatabricksUCDatasource schema() shadowing by schema string attribute ( #61282 ) Fix AliasExpr structural equality to respect rename flag ( #60711 ) Fix _align_struct_fields failure with unaligned scalar fields ( #58364 ) Fix min_scheduling_resources fallback to incremental_resource_usage ( #60997 ) Fix output backpressure unblocking sequence for terminal ops ( #60798 ) Fix multi-input operator object store memory attribution ( #61208 ) Fix reference cycle by moving to module scope ( #61934 ) Fix autoscaler logging: reduce verbose output and move traceback to debug ( #61989 , #62126 ) Fix double counting ref_bundle + input_files ( #61774 ) Replace on_exit hook with __ray_shutdown__ to fix UDF cleanup race ( #61700 ) Prevent Limit from getting pushed past map_groups ( #60881 ) Propagate schema in empty _shuffle_block to fix ColumnNotFound in chained left joins ( #61507 ) Fix unclear metadata warning and incorrect operator name logging ( #61380 ) Clamp rolling utilization averages to zero ( #61543 ) Fix floating point errors in TimeWindowAverageCalculator ( #61580 ) Remove default task-level timeout and clamp end_offset in Kafka datasource ( #61476 ) Avoid redundant reads in train_test_split ( #60274 ) Return None when no outputs have been produced ( #62029 ) Replace bare raise with TypeError in string concatenation ( #60795 ) 📖 Documentation Add job-level checkpointing documentation ( #60921 ) Update exclude_resources docs for Train autoscaling changes ( #61990 ) Add locality_with_output migration instructions ( #61151 ) Document max_tasks_in_flight_per_actor vs max_concurrent_batches ( #60477 ) Add missing MOD operation docs; improve ray.data.Datasource docs ( #60803 , #59654 ) Add polars usage instructions ( #60029 ) Ray Serve 🎉 New Features: Added end-to-end gRPC client and bidirectional streaming support, including public APIs, proxy handling, proto updates, and developer docs, so Serve apps can handle streaming workloads natively instead of building custom transport layers. ( #60767 , #60768 , #60769 , #60770 , #60771 ) Introduced HAProxy-based serving with fallback proxy support and load-balancer tunables, giving operators a higher-throughput ingress path and more control over traffic behavior in production. ( #60586 , #61180 , #61271 , #61468 , #61988 ) Added queue-based autoscaling for async inference and Taskiq-backed workloads, so scaling decisions can account for both HTTP in-flight load and queued tasks. ( #59548 , #60851 , #60977 , #61008 ) Rolled out gang scheduling support across validation, core scheduling, fault tolerance, downscaling, autoscaling, rolling updates, and migration, enabling coordinated multi-replica placement for tightly coupled workloads. ( #60944 , #61205 , #61206 , #61207 , #61215 , #61467 , #61216 , #61659 ) Introduced deployment-scoped actors with config/schema, lifecycle management, public API, and controller health checks, making it easier to run durable per-deployment sidecar-like logic inside Serve. ( #61639 , #61648 , #61664 , #61833 , #62161 ) 💫 Enhancements: Added first-class tracing support for Serve, including inter-deployment gRPC propagation and richer streaming-path attributes, improving end-to-end observability across distributed request flows. ( #61230 , #61089 , #61451 ) Expanded operational metrics with replica utilization, richer error labeling, and client IP logging in access logs, helping teams diagnose bottlenecks and user-impacting issues faster. ( #60758 , #61092 , #60967 ) Improved autoscaling extensibility with class-based policies and policy_kwargs , so advanced users can package reusable autoscaling logic without custom forks. ( #60964 ) Reduced controller overhead with broad algorithmic improvements (indexing, cache reuse, and avoiding repeated per-tick work), which improves scalability as deployment and replica counts grow. ( #60810 , #60829 , #60830 , #60838 , #60842 , #60843 , #60844 , #60832 , #60806 ) Improved throughput-oriented operation controls by adding environment-based tuning and explicit throughput optimization logging, making performance behavior easier to configure and audit. ( #60757 , #62146 ) Upgraded Serve internals to Pydantic v2 and refined time-series aggregation behavior for more predictable metric accuracy under high load. ( #61061 , #61403 ) 🔨 Fixes: Fixed a direct-ingress shutdown bug where replicas could hang indefinitely while draining stuck requests, ensuring bounded shutdown behavior in failure scenarios. ( #60754 ) Fixed HAProxy reliability issues, including config race conditions, draining guards, and platform compatibility edge cases, improving stability in production rollouts. ( #61120 , #60955 ) Fixed autoscaling correctness issues that could cause runaway scaling or delayed reactions, including feedback-loop regressions, streaming scale-down behavior, and wall-clock delay handling. ( #61731 , #61920 , #62331 , #61844 , #60613 ) Fixed high-percentile latency regression in request routing and queue-length accounting, reducing tail-latency spikes under load. ( #61755 ) Fixed replica-state and health-state edge cases during migration and ingress transitions, preventing false errors and unhealthy/healthy misreporting. ( #60365 , #61818 , #62213 ) Fixed chained upstream actor-failure handling so request failures are attributed correctly and no longer hang when upstream deployments die mid-chain. ( #61758 , #62147 ) Fixed HTTP status classification for client disconnects after successful responses, improving accuracy of error-rate monitoring and alerting. ( #61396 ) 📖 Documentation: Added AsyncInferenceAutoscalingPolicy documentation and clarified Serve performance guidance for HAProxy and inter-deployment gRPC use cases. ( #61086 , #61386 ) Updated scheduling and configuration docs, including replica scheduling guidance and a catalog of Serve environment variables, so operators can tune deployments with less guesswork. ( #60922 , #60807 ) Clarified multiplexing and async behavior docs (including model pre-warming constraints and request-cancel semantics) to prevent common integration mistakes. ( #61842 , #62280 ) 🏗 Architecture refactoring: Refactored deployment-state execution to skip unnecessary steady-state per-tick work, lowering control-loop churn and creating cleaner hooks for future scheduling logic. ( #60840 ) Moved autoscaling metric aggregation into Cython-backed paths and added focused controller benchmarking, giving a stronger performance baseline for future Serve controller changes. ( #58892 , #61368 ) Simplified internal structure by migrating shared internals away from private modules and consolidating replica abstractions, reducing coupling and maintenance complexity. ( #60849 , #61363 , #60198 ) Ray Train 🎉 New Features Elastic training: core capability, user guide, release tests, multi-host TPU, telemetry ( #60721 , #61115 , #61133 , #61299 , #61267 ) Add HF TRL (Transformer Reinforcement Learning) example ( #61627 ) Add Tensor Parallel templates for DeepSpeed AutoTP and DTensor ( #60160 , #60158 ) Add status attribute to ReportedCheckpoint ( #61684 ) Richer Train run metadata ( #59186 ) Add timers for Train worker initialization ( #60870 ) Configure torchft environment ( #61156 ) 💫 Enhancements Register training resources with AutoscalingCoordinator in FixedScalingPolicy ( #61703 ) Decouple datasets field from TrainRunContext ( #61953 ) Log warning for checkpoint_upload_fn when slow ( #61720 ) Fix StateManagerCallback to accept datasets explicitly ( #62042 ) Make train run abortable during before_controller_shutdown ( #61816 ) Graceful abort catches all RayActorError ( #61375 ) Refactor checkpoint and sync_actor to use wait_with_logging ( #61063 ) Unwrap UserExceptionWithTraceback in WorkerGroupError.worker_failures ( #61153 ) 🔨 Fixes Fix v2 PlacementGroupCleaner zombie actor ( #61756 ) Fix checkpoint paths for multinode run ( #61471 ) Abort cancels validation tasks with deterministic resumption ( #61510 ) Fix deepspeed finetune release test ( #61266 ) 📖 Documentation Add section on async validation with experiment tracking ( #62104 ) Add section on when to use async validation ( #61702 ) Ray Tune 💫 Enhancements Remove deprecated Logger interface and logger_creator ( #61181 ) 🔨 Fixes Fix PBT trial order when NaN values are present ( #57160 ) Ray LLM 🎉 New Features Replace PDProxyServer with decode-as-orchestrator PD architecture ( #62076 ) Introduce DP group fault tolerance for WideEP deployments ( #61480 ) SGLang engine: streaming chat/completions, tokenize/detokenize, embeddings, multi-GPU TP/PP ( #61236 , #61446 , #61159 , #61201 , #62221 ) Add bundle_per_worker config for simpler placement group setup ( #59903 ) Separate Data and Serve LLM dashboards with improved panel visibility ( #61037 , #62069 ) 💫 Enhancements Promote Data LLM and Serve LLM APIs to beta ( #61249 , #62054 , #62223 ) Upgrade vLLM to 0.16.0, 0.17.0, and 0.18.0 ( #61389 , #61598 , #61952 ) Upgrade NIXL to v1.0.0 and fix tensor transport issues ( #61991 ) Unify duplicated PlacementGroup config schemes ( #62241 ) Decouple Serve LLM ingress from vLLM protocol models ( #61931 ) Set download task num_cpus=0 to reduce contention on low-CPU machines ( #61191 ) SGLangServer cleanup and replace format_messages_to_prompt with _build_chat_messages ( #61117 , #61372 ) 🔨 Fixes Fix duplicate data: [DONE] in streaming SSE responses ( #62246 ) Fix enable_log_requests=False not forwarded to vLLM AsyncLLM ( #60824 ) Fix OpenAiIngress scale-to-zero when all models set min_replicas=0 ( #60836 ) Handle missing state attributes from vLLM's task-conditional init_app_state ( #60812 ) Fix NIXL side channel host for cross-node P/D disaggregation ( #60817 ) Fix trust_remote_code download ( #60344 ) Avoid deprecated TRANSFORMERS_CACHE ; treat HuggingFace config load failure as non-fatal ( #60854 ) Fix sequential batch processing in SGLangServer ( #61189 ) 📖 Documentation Update data parallel attention documentation ( #61706 ) Add custom tokenizer example ( #61098 ) Add C/C++ binaries incompatibility workaround ( #62110 ) Ray RLlib 💫 Enhancements Connector/batching optimizations: ndarray fast paths, direct env step pipeline, batch reuse ( #61320 , #61255 , #61256 , #61259 , #61144 ) Unify default encoders for all algorithms ( #60302 ) Toggle eval/train mode in TorchRLModule forward passes ( #61985 ) Clean up offline prelearner and unit testing ( #60632 ) Remove duplicate assignments in AlgorithmConfig ( #61233 ) Remove legacy RLlib release tests ( #59288 ) Add APPO example with Footsies environment ( #59006 ) 🔨 Fixes Support custom eval functions returning zero eval_results , env_steps , or agent_steps ( #61563 ) Fix PrioritizedEpisodeReplayBuffer bug ( #60065 ) Fix missing LayerNorm in RLModuleSpec ( #61025 ) Fix evaluation in parallel to training ( #60777 ) Fix MultiAgentEpisode.env_t_to_agent_t ( #60319 ) Fix default metric during eval ( #61590 ) Fix incorrect log value of environment steps sampled/trained ( #56599 ) Prevent torch_learner.py crash under parameter-freezing edge cases ( #62158 ) Ray Core 🎉 New Features Resource isolation: pressure-based memory monitor, time-based killing, cgroup constraints ( #61361 , #61323 , #61097 , #61210 , #61297 , #59365 , #59368 , #60752 ) IPPR: add ResizeRayletResourceInstances to GCS/Python client, schema/status models, KubeRay provider ( #61654 , #61666 , #61803 , #61814 ) Add PlatformEvent proto and placement group events in one-event framework ( #61701 , #60449 ) Add Nvidia B300 support ( #60753 ) Add UV support for Ray Client mode ( #60868 ) Add Percentile metric type backed by quadratic histogram ( #61148 ) Expose fallback_strategy in TaskInfoEntry and ActorTableData ( #60659 ) Add submission job proto changes ( #60857 ) Add TPU util for ready multi-host slice count; simplify elastic TPU scaling ( #61300 , #62141 ) Introduce per-node level temp-dir ( #60761 ) Make ray.put() generic: put(value: R) -> ObjectRef[R] ( #60995 ) Add Python 3.14 support for recursion limit handling ( #58459 ) 💫 Enhancements Upgrade cloudpickle to 3.1.2, gRPC to v1.58.0, protobuf to 3.20.3 ( #60317 , #61499 , #60736 ) Multiple gRPC connections for improved object transfer throughput, enabled by default ( #61121 , #61440 ) Improve pg.ready() performance via async GCS RPC; fix deadlocks ( #60657 , #62086 ) RDT: non-torch transfers, PyTorch storage caching, metadata caching, NIXL agent reuse ( #61081 , #60999 , #60689 , #60602 ) Cache ActorHandle.__hash__ and fix __eq__ correctness ( #61638 ) Cache find_gcs_addresses ( #61065 ) Optimize worker listener thread ( #61353 ) Eliminate Python GCS client from state manager get_all_node_info ( #61232 ) Loosen restriction on worker thread count ( #62279 ) Sequence in-order actor tasks per concurrency group instead of globally ( #61082 ) Prioritize killing workers that occupy large memory in OOM killer ( #60330 ) Cap exponential backoff attempt number to prevent integer overflow ( #61003 ) Replace deprecated threading APIs ( getName / setDaemon ) ( #62153 ) Improve error handling for @ray.remote / @ray.method with num_returns ( #59286 ) Convert StopIteration on non-generator functions to RuntimeError ( #60521 ) Surface warnings for scheduling rate limits slowing task ramp-up ( #61004 ) Periodically reload service account tokens; use AuthenticationValidator in sync server ( #60778 , #60779 ) Remove support for local_mode ( #60647 ) Allow matching worker_process_setup_hook on re-entry ( #61473 ) Reduce default event aggregator buffer size to avoid OOM ( #60826 ) Suppress autoscaler action logs for read-only provider ( #61732 ) Lazy subscription to node changes on non-driver workers ( #61118 ) Tighten export symbol allowlists to prevent non-ray symbol leakage ( #61298 ) Approximate USS from memory_info instead of calling memory_full_info ( #60000 ) Dedicated IO context for NodeManager and InternalKVManager ( #61002 ) Print gRPC peer address on GCS HandleUnregisterNode / HandleDrainNode ( #62226 , #62112 ) 🔨 Fixes Fix task stuck when pop worker repeatedly fails ( #60104 ) Fix bool env var parsing for RAY_CGRAPH_overlap_gpu_communication ( #61421 ) Fix negative RUNNING task metric ( #62070 ) Fix OnNodeDead to destroy all owned actors when owner node dies ( #60669 ) Fix actor task queue blocked after cancelling head task ( #60850 ) Fix TASK_PROFILE_EVENT aggregation for multiple phases ( #61559 ) Fix double-counting in WorkerPool::WarnAboutSize() ( #61246 ) Fix TaskLifecycleEvent.node_id using emitting node instead of executor ( #61478 ) Fix publisher_id type mismatch in GCS pubsub ( #61518 ) Fix dataclass.asdict with None in dashboard list_jobs API ( #61033 ) Fix dashboard node head API dead node cache ( #61185 ) Fix dashboard event agent for events without HTTP scheme ( #60811 ) Fix Ray Actor typing for async methods ( #60682 ) Fix autoscaler retry during k8s exceptions ( #60658 ) Fix ReadOnlyProvider.terminate() signature mismatch ( #62251 ) Fix set / get env races in OtlpGrpcMetricExporterOptions and metrics exporter init ( #61034 , #61281 ) Clean up node processes on version mismatch during ray start ( #61837 ) Retry node discovery upon ray.init() ( #61029 ) Ensure Node._node_labels initializes regardless of connect_only ( #61618 ) Avoid reentrant locking in worker context ( #61925 ) Java Local Mode type confusion with multiple Actor types ( #61858 ) Recover from WrongClusterID on head restart ( #60860 ) Fix Azure: do not delete shared MSI when tearing down clusters ( #61811 ) Configure TLS/mTLS for OpenTelemetry OTLP gRPC exporter ( #60745 ) Dashboard 🎉 New Features Add Queued Blocks metric to Ray Data Dashboard ( #61716 ) Add Logical Memory Usage panel ( #60772 ) Add running tasks by node, update Ray Data active tasks panel ( #61641 ) Add NIXL KV transfer metrics to Serve LLM Grafana dashboard ( #60819 ) Add GPU power and temperature graphs ( #60942 ) Support log links in Grafana dashboard ( #60896 ) Support autoscaler v2 for cluster-level node metrics ( #60504 ) Add middleware proxy for history server ( #61295 ) Forward **kwargs through JobSubmissionClient to cluster info resolvers ( #61902 ) Ray Wheels and Images Upgrade Bazel from 6.5.0 to 7.5.0 ( #61601 ) Bump torch to 2.7.0+cu128 and torchvision ( #61328 ) Upgrade jackson-databind 2.16.1 -> 2.18.6 ( GHSA-72hv-8253-57qq ) ( #61808 ) Upgrade CI containers from Ubuntu 20.04 to 22.04; Forge from clang-12 to clang-14 ( #61533 , #61662 ) Add CUDA 13 images for ray-llm/core-gpu and release test configs ( #61497 , #61637 ) Add py312+CUDA 12.9 and py312+CUDA 13 depsets for Ray LLM ( #61116 , #61149 , #61496 ) Add TPU Docker images to CI build and publish pipeline ( #61172 , #61173 , #61174 , #61175 ) Add Python 3.14 to Linux wheel verification ( #62127 ) Windows base build fix ( #62415 ) Add build-image.sh and CLI for local Docker image builder ( #61042 , #61338 ) Support repeated execution of setup-dev.py ( #61357 ) Documentation Add Ray History Server user guide ( #62030 ) Add RAY_BACKEND_LOG_JSON environment variable documentation ( #59962 ) Add user guide for Ray token auth with Kubernetes RBAC ( #61644 ) Add warning about token authentication in untrusted networks ( #62248 ) KubeRay: prerunning deadline docs, version 1.6.0 refs, GKE/cgroups cross-refs ( #61552 , #61865 , #62140 ) Use RayCluster name as ServiceAccount name for RBAC authentication ( #61785 ) Remove outdated note on labels in local RayCluster ( #61719 ) List TPUs as fully tested/supported ( #61634 ) Restructure development.rst with image build, wheel paths, and cross-references ( #61500 , #61501 , #61504 , #61596 ) Remove incorrect warning for placement groups ( #61176 ) Add multi-agent A2A example ( #61193 ) Add object spill internal doc ( #60930 ) Thanks Many thanks to all those who contributed to this release! @justinyeh1995 , @marwan116 , @jddqd , @MkDev11, @mjd3 , @XuQianJin-Stars , @elliot-barn , @DeborahOlaboye , @aaronscalene , @rayhhome , @ayushk7102 , @bj-son , @nadongjun , @Daraan , @xinyuangui2 , @Sparks0219 , @justinvyu , @suppagoddo , @akyang-anyscale , @ambicuity , @Aydin-ab , @mickeyyliu , @MatthewCWeston , @vaishdho1 , @jinbum-kim , @eicherseiji , @kouroshHakha , @karticam , @JasonLi1909 , @ArturNiederfahrenhorst , @moktamd , @nrghosh , @dragongu , @andrewsykim , @mgchoi239 , @ruoliu2 , @harshit-anyscale , @Chong-Li , @pseudo-rnd-thoughts , @lee1258561 , @khluu , @daiping8 , @SolitaryThinker , @jonalee99, @yancanmao , @SohamRajpure , @rueian , @VitaliyEroshin , @Future-Outlier , @nehiljain , @JiangJiaWei1103 , @Yicheng-Lu-llll , @KaisennHu , @jeffreywang-anyscale, @aslonnie , @alanwguo , @machichima , @limarkdcunha , @codope , @sampan-s-nayak , @kyuds , @thjung123 , @abrarsheikh , @wingkitlee0 , @preneond , @7ckingBest , @slfan1989 , @win5923 , @kaori-seasons , @israbbani , @andrew-anyscale , @zestze , @owenowenisme , @edoakes , @laysfire , @pushpavanthar , @tohtana , @leewyang , @liulehui , @Hyunoh-Yeo , @eureka0928, @ryanaoleary , @947132885 , @Kunchd , @simonsays1980 , @dpj135 , @bveeramani , @raulchen , @Partth101 , @dubin555 , @richabanker , @bittoby, @sai-miduthuri , @RedGrey1993 , @kamil-kaczmarek , @TimothySeah , @myandpr , @rishic3 , @justinrmiller , @HassamSheikh , @chiayi , @petern48 , @carolynwang , @MrKWatkins , @400Ping , @summaryzb , @peterxcli , @RocMarshal , @coqian , @yuhuan130 , @ryankert01 , @dayshah , @Anarion-zuo , @ZacAttack , @weimingdiit , @iamjustinhsu , @matthewdeng , @goutamvenkat-anyscale , @KeeProMise , @Sanskarzz , @yuchen-ecnu , @praneethkaturi , @rajeshg007 , @ankur-anyscale , @Art0white , @xyuzh , @dancingactor , @MengjinYan , @dengkliu92 , @alexeykudinkin

Ray Data 🔨 Fixes Disable hanging issue detection ( #61895 ) — The hanging issue detector was making blocking calls to the Ray State API, which could cause the scheduling loop to block and severely degrade pipeline performance. The detector is disabled in this patch release until the blocking calls are fixed.

Ray Data 🎉 New Features Add checkpointing support to Ray Data ( #59409 ) Compute Expressions: list operations ( #59346 ), fixed-size arrays ( #58741 ), string padding ( #59552 ), logarithmic ( #59549 ), trigonometric ( #59712 ), arithmetic ( #59678 ), and rounding ( #59295 ) Add sql_params support to read_sql ( #60030 ) Add AsList aggregation ( #59920 ) Support CountDistinct aggregate ( #59030 ) Add credential provider abstraction for Databricks UC datasource ( #60457 ) Support callable classes for UDFExpr ( #56725 ) Add autoscaler metrics to Data Dashboard ( #60472 ) Add optional filesystem parameter to download expression ( #60677 ) Allow specifying partitioning style or flavor in write_parquet() ( #59102 ) New cluster autoscaler enabled by default ( #60474 ) 💫 Enhancements Improve numerical stability in scalers by handling near-zero values ( #60488 ) Export dataset operator output schema to event logger ( #60086 ) Iceberg: add retry policy for Storage + Catalog writes ( #60620 ) Iceberg: remove calls to Catalog Table in write tasks ( #60476 ) Expose logical operators and rules via package exports ( #60297 , #60296 ) Demote Sort from requiring preserve_order ( #60555 ) Improve appearance of repr(dataset) ( #59631 ) Allow configuring DefaultClusterAutoscalerV2 thresholds via env vars ( #60133 ) Use Arrow IPC for Arrow Schema serialization/deserialization ( #60195 ) Store _source_paths in object store to prevent excessive spilling during read task serialization ( #59999 ) Add more shuffle fusion rules ( #59985 ) Enable and tune DownstreamCapacityBackpressurePolicy ( #59753 ) Enable concurrency cap backpressure with tuning ( #59392 ) Set default actor pool scale up threshold to 1.75 ( #59512 ) Don't downscale actors if the operator hasn't received any inputs ( #59883 ) Don't reserve GPU budget for non-GPU tasks ( #59789 ) Only return selected data columns in hive-partitioned Parquet files ( #60236 ) Ordered + FIFO bundle queue ( #60228 ) Add node_id , pid , attempt number for hanging tasks ( #59793 ) Revise resource allocator task scheduling to factor in pending task outputs ( #60639 ) Track block serialization time ( #60574 ) Use metrics from OpRuntimeMetrics for progress ( #60304 ) Tabular form for streaming executor op metrics ( #59774 ) Info-log cluster scale-up decisions ( #60357 ) Use plain mode instead of grid mode for OpMetrics logging ( #59907 ) Progress reporting refactors ( #59350 , #59629 , #59880 ) Remove deprecated TENSOR_COLUMN_NAME constant ( #60573 ) Remove meta_provider parameter ( #60379 ) Decouple Ray Train from Ray Data by removing top-level ray.data imports ( #60292 ) Move extension types to ray.data ( #59420 ) Skip upscaling validation warning for fixed-size actor pools ( #60569 ) Make StatefulShuffleAggregation.finalize allow incremental streaming ( #59972 ) Revisit OutputSplitter semantics to avoid unnecessary buffer accumulation ( #60237 ) Update to PyArrow 23 ( #60739 , #59489 ) Add BackpressurePolicy to streaming executor progress bar ( #59637 ) Support Arrow-based transformations for preprocessors ( #59810 ) StandardScaler preprocessor with Arrow format ( #59906 ) OneHotEncoder with Arrow format ( #59890 ) 🔨 Fixes Fuse MapBatches even if they modify the row count ( #60756 ) Don't push limit past map_batches by default ( #60448 ) Fix wrong type hint of other dataset in zip and union ( #60653 ) Fix ActorPoolMapOperator to guarantee dispatch of all given inputs ( #60763 ) Fix ArrowInvalid error when backfilling missing fields from map tasks ( #60643 ) Fix attribute error in UnionOperator.clear_internal_output_queue ( #60538 ) Fix DefaultClusterAutoscalerV2 raising KeyError: 'CPU' ( #60208 ) Fix ReorderingBundleQueue handling of empty output sequences ( #60470 ) Fix task completion time without backpressure grafana panel metric name ( #60481 ) Fix Union operator blocking when preserve_order is set ( #59922 ) Fix autoscaler requesting empty resources instead of previous allocation when not scaling up ( #60321 ) Fix autoscaler not respecting user-configured resource limits ( #60283 ) Fix DefaultAutoscalerV2 not scaling nodes from zero ( #59896 ) Fix Iceberg warning message ( #60044 ) Fix Parquet datasource path column support ( #60046 ) Fix ProgressBar with use_ray_tqdm ( #59996 ) Fix stale stats on refit for preprocessors ( #60031 ) Fix StreamingRepartition hang with empty upstream results ( #59848 ) Fix operator fusion bug to preserve UDF modifying row count ( #59513 ) Fix AutoscalingCoordinator double-allocating resources for multiple datasets ( #59740 ) Fix DownstreamCapacityBackpressurePolicy issues ( #59990 ) Fix AutoscalingCoordinator crash when requesting 0 GPUs on CPU-only cluster ( #59514 ) Fix TensorArray to Arrow tensor conversion ( #59449 ) Fix resource allocator not respecting max resource requirement ( #59412 ) Fix GPU autoscaling when max_actors is set ( #59632 ) Fix checkpoint filter PyArrow zero-copy conversion error ( #59839 ) Restore class aliases to fix deserialization of existing datasets ( #59828 , #59818 ) Fix DataContext deserialization issue with StatsActor ( #59471 ) 📖 Documentation Sort references in "Loading data and Saving data" pages ( #60084 ) Fix inconsistent heading levels in "How to write tests" guide ( #60706 ) Clarify resource_limits refers to logical resources ( #60109 ) Update read_lance doc ( #59673 ) Fix broken link in read_unity_catalog docstring ( #59745 ) Fix bug in docs for enable_true_multi_threading ( #60515 ) Add more education around transformations ( #59415 ) Ray Serve 🎉 New Features Queue-based autoscaling for TaskConsumer deployments (phase 1). Introduces a QueueMonitor actor that queries message brokers (Redis, RabbitMQ) for queue length, enabling TaskConsumer scaling based on pending tasks rather than HTTP load. ( #59430 ) Default autoscaling parameters for custom policies. New apply_autoscaling_config decorator allows custom autoscaling policies to automatically benefit from Ray Serve's standard parameters (delays, scaling factors, bounds) without reimplementation. ( #58857 ) label_selector and bundle_label_selector in Serve deployments. Deployments can now specify node label selectors for scheduling and bundle-level label selectors for placement groups, useful for targeting specific hardware (e.g., TPU topologies). ( #57694 ) Deployment-level autoscaling observability. The controller now emits a structured JSON serve_autoscaling_snapshot log per autoscaling-enabled deployment each control-loop tick, with an event summarizer that reduces duplicate logs. ( #56225 ) Batching with multiplexing support. Batching now guarantees each batch contains requests for the same multiplexed model, enabling correct multiplexed model serving with @serve.batch . ( #59334 ) 💫 Enhancements Replica routing data structure optimizations. O(1) pending-request lookups, cached replica lists, lazy cleanup, optimized retry insertion, and metrics throttling yield significant routing performance improvements. ( #60139 ) New operational metrics suite. Added long-poll metrics, replica lifecycle metrics, app/deployment status metrics, proxy health and request routing delay metrics, event loop utilization metrics, and controller health metrics — greatly improving monitoring and debugging capabilities. ( #59246 , #59235 , #59244 , #59238 , #59535 , #60473 ) Autoscaling config validation. lookback_period_s must now be greater than metrics_interval_s , preventing silent misconfigurations. ( #59456 ) Cross-version root_path support for uvicorn. root_path now works correctly across all uvicorn versions, including >=0.26.0 which changed how root_path is processed. ( #57555 ) Preserve user-set gRPC status codes. When deployments raise exceptions after setting a gRPC status code on the context, that code is now correctly propagated to the client instead of being overwritten with INTERNAL. Error messages are truncated to 4 KB to respect HTTP/2 trailer limits. ( #60482 ) Replica ThreadPoolExecutor capped to num_cpus. The user-code event loop's default ThreadPoolExecutor is now limited to the deployment's num_cpus, preventing oversubscription when using asyncio.to_thread. ( #60271 ) Generic actor registration API for shutdown cleanup. Deployments can register auxiliary actors (e.g., PrefixTreeActor) with the controller for automatic cleanup on serve.shutdown() , eliminating cross-library import dependencies. ( #60067 ) Deployment config logging in controller. Deployment configurations are now logged in the controller for easier debugging and auditability. ( #59222 , #59501 ) Pydantic v1 deprecation warning. A FutureWarning is now emitted at ray.init() when Pydantic v1 is detected, as support will be removed in Ray 2.56. ( #59703 ) 🔨 Fixes Fixed tracing signature mismatch across processes. Resolved TypeError: got an unexpected keyword argument _ray_trace_ctx when calling actors from a different process than the one that created them (e.g., serve start + dashboard interaction). ( #59634 ) Fixed ingress deployment name collision. Ingress deployment name was incorrectly modified when a child deployment shared the same name, causing routing failures. ( #59577 ) Fixed downstream deployment over-provisioning. Downstream deployments no longer over-provision replicas when receiving DeploymentResponse objects. ( #60747 ) Fixed replicas hanging forever during draining. Replicas no longer hang indefinitely when requests are stuck during the draining phase. ( #60788 ) Fixed TaskProcessorAdapter shutdown during rolling updates. Removed shutdown() from __del__ , which was broadcasting a kill signal to all Celery workers instead of just the local one, breaking rolling updates. ( #59713 ) Fixed Windows test failures. Resolved tracing file handle cleanup on Windows, skipped incompatible gRPC and tracing tests on Windows. ( #60078 , #60356 , #60393 , #59771 ) Fixed flaky tests. Addressed gauge throttling race in test_router_queue_len_metric , ensured proxy replica queue cache is populated before GCS failure tests, and added metrics server readiness checks. ( #60333 , #60466 , #60468 ) Fixed distilbert test segfault. Worked around a pyarrow/jemalloc crash triggered by specific import ordering of FastAPI, torch, and TensorFlow. ( #60478 ) 📖 Documentation Improved autoscaling documentation. Clarified the relationship between delays, metric push intervals, and the autoscaling control loop. ( #59475 ) New example: video analysis inference. End-to-end notebook demonstrating a Serve application for scene change detection, - tagging, and video description. ( #59859 ) New examples: model multiplexing and model composition. Published workload-based examples for forecasting with model multiplexing and recommendation systems with model composition. ( #59166 ) Model registry integration guide. Added documentation for integrating Serve with model registries (e.g., MLflow). ( #59080 ) Fixed broken documentation links. Resolved 404 errors for async inference, MLflow registry example, and LLM code examples. ( #59917 , #60071 , #59520 , #59521 , #60181 ) Fixed monitoring docs. Corrected target replicas metric emission to enable time-series comparison with actual replicas. ( #59571 ) Async inference template. Added an end-to-end template for building asynchronous inference applications with Ray Serve. ( #58393 , #59926 ) 🏗 Architecture refactoring Environment variable cleanup (5-part series). Removed deprecated and redundant env vars ( RAY_SERVE_DEFAULT_HTTP_HOST , RAY_SERVE_DEFAULT_HTTP_PORT , RAY_SERVE_DEFAULT_GRPC_PORT , RAY_SERVE_HTTP_KEEP_ALIVE_TIMEOUT_S , RAY_SERVE_REQUEST_PROCESSING_TIMEOUT_S , RAY_SERVE_ENABLE_JSON_LOGGING , RAY_SERVE_ALWAYS_RUN_PROXY_ON_HEAD_NODE ), cleaned up legacy constant fallbacks, and added documentation for previously undocumented env vars (e.g., RAY_SERVE_CONTROLLER_MAX_CONCURRENCY , RAY_SERVE_ROOT_URL , proxy health check settings, and fault tolerance params). Users relying on removed env vars should migrate to the Serve config API ( http_options , grpc_options , LoggingConfig ). ( #59470 , #59619 , #59647 , #59963 , #60093 ) Ray Train 🎉 New Features Add TPU multi-slice support to JaxTrainer ( #58629 ) Update async validation API ( #59428 ) Add a CallbackManager and guardrail some callback hooks ( #60117 ) Add inter-execution file shuffling for deterministic multi-epoch training ( #59528 ) Resume validations on driver restoration ( #59270 ) 💫 Enhancements Pass ray remote args to validation task ( #60203 ) Deprecate Predictor API ( #60305 ) Increase worker group start default timeout to 60s ( #60376 ) Unify PlacementGroup and SlicePlacementGroup interface in WorkerGroup ( #60116 ) Cleanup zombie RayTrainWorker actors ( #59872 ) Add usage telemetry for checkpointing and validation ( #59490 ) Validate that validation is called with a checkpoint ( #60548 ) Replace pg.ready() with pg.wait() in worker group ( #60568 ) Rename DatasetsSetupCallback to DatasetsCallback ( #59423 ) Update "Checkpoint Report Time" metric title to "Cumulative Checkpoint Report Time" ( #58470 ) Add training failed error back to failure policy log ( #59957 ) Decouple Ray Train from Ray Data by removing top-level imports ( #60292 ) 🔨 Fixes Add try - except for pg.wait() ( #60743 ) TrainController reraises AsyncioActorExit ( #59461 ) 📖 Documentation Add a JaxTrainer template ( #59842 ) Update Jax doc to include GPU and multi-slice TPU support ( #60593 ) Document checkpoint_upload_fn backend and cuda:nccl backend support ( #60541 ) Rename checkpoint_upload_func to checkpoint_upload_fn in docs ( #60390 ) Fix Ray Train workloads and PyTorch with ASHA templates ( #60537 ) Publish Ray Train workload example ( #58936 ) Ray Tune 🔨 Fixes Avoid file deletion race by using unique tmp file names ( #60556 ) Ray LLM 🎉 New Features Add /tokenize and /detokenize endpoints ( #59787 ) Add /collective_rpc endpoint for RLHF weight synchronization ( #59529 ) Add Control Plane API for Sleep/Wakeup ( #59455 ) Add Pause/Resume Control Plane API ( #59523 ) Add support for classification and scoring models ( #59499 ) Add pooling parameter ( #59534 ) Support vLLM structured outputs with backward-compat for guided_decoding ( #59421 ) Add CPU support to Ray Serve LLM ( #58334 ) Add should_continue_on_error support for ServeDeploymentStage ( #59395 ) Support configuring HttpRequestUDF resources ( #60313 ) 💫 Enhancements Upgrade vLLM to 0.15.0 ( #60679 ) Unify schema of success and failure rows ( #60572 ) Prefer uniproc executor over mp executor when world_size==1 ( #60403 ) Use compute instead of concurrency to specify ActorPool size ( #59645 ) Remove DataContext overrides in Ray Data LLM Processor ( #60142 ) Use numpy arrays for embeddings to avoid torch.Tensor serialization overhead ( #59919 ) Make PrefixCacheAwareRouter imbalance threshold less surprising ( #59390 ) Allow tokenized_prompt without prompt in vLLMEngineStage ( #59801 ) Avoid passing enums through fn_constructor_kwargs ( #59806 ) Refactor Control Plane endpoints into mixins ( #59502 ) Remove CUDA_VISIBLE_DEVICES deletion workaround ( #60502 ) 🔨 Fixes Fix nested dict to Namespace conversion in vLLM engine initialization ( #60380 ) Fix JSON non-serializable ndarray exception in http_request_stage ( #60299 ) Exit actor on EngineDeadError to enable recovery ( #60145 ) Fix NIXL port conflict in prefill-decode disaggregation test ( #60057 ) 📖 Documentation Batch inference docs reorg and update to reflect per-stage config refactor ( #59214 ) Add resiliency section and refine doc code ( #60594 ) Add video/audio examples for vLLMEngineProcessor ( #59446 ) Add SGLang integration example ( #58366 ) Remove inaccurate statement in docs ( #60425 ) Ray RLlib 🎉 New Features Add TQC (Truncated Quantile Critics) algorithm implementation ( #59808 ) Add LR scheduling ability to BC and MARWIL ( #59067 ) RLlib and Ray Tune: Hyperparameter Optimisation example ( #60182 ) 💫 Enhancements 🔥 APPO improvements: learner pipeline performance improvements ( #59544 ) Improve stateful model training on offline data ( #59345 ) Create resource bundle per learner ( #59620 ) Improve env runner sampling by replacing recursive solution with iterative solution ( #56082 ) Improve IMPALA examples and premerge ( #59927 ) Remove MLAgents dependency ( #59524 ) Upgrade to gymnasium v1.2.2 ( #59530 ) Decrease log quantity for learning tests ( #59005 ) Update learner state warnings to the debug level ( #60178 ) Don't log np.nanmean warnings in EMA stats ( #60408 ) 🔨 Fixes Fix DQN RLModule forward methods to handle dict spaces ( #60451 ) Fix LearnerGroup.load_module_state() and mark as deprecated ( #60354 ) Fix static dimension issue in ONNX export of Torch attention models ( #60102 ) Fix Multi-Agent Episode concatenation for sequential environments ( #59895 ) Fix module episode returns metrics accumulation for shared module IDs ( #60234 ) Fix rollout fragment length calculation in AlgorithmConfig ( #59438 ) Fix checkpointable issues with cloud storages ( #60440 ) Update flatten_observations.py for nested spaces for ignored multi-agent ( #59928 ) Ray Core 🎉 New Features Resource Isolation: unify config construction, add public docs, and expose cgroup_path in ray.init() ( #59372 , #60183 , #60726 ) Support tensor-level deduplication for NIXL ( #60509 ) Add CUDA IPC transport for RDT ( #59838 ) Register custom transport at runtime for RDT ( #59255 ) Support TPU v7x accelerator type for device discovery ( #60338 ) Introduce local port service discovery ( #59613 ) Cancel sync actor by checking is_canceled() ( #58914 ) Support labels for ray job submit --entrypoint-resource ( #59735 ) Add --ip option in ray attach ( #59931 ) Add bearer token support for remote URI downloads ( #60050 ) Support HTTP redirection download ( #59384 ) Add ray kill-actor --name/--namespace for force/graceful shutdown ( #60258 ) 💫 Enhancements Bound object spilling file size to avoid disk increase pressure ( #60098 ) Replace SHA-1 with SHA-256 for internal hash operations ( #60242 ) Use whitelist approach to block mutation requests from browser ( #60526 ) Pass authentication headers to WebSocket connections in tail_job_logs ( #60346 ) Add auth to Dashboard HTTP agent and client ( #59891 ) Use dedicated service account path for Ray auth tokens ( #60409 ) Update Kubernetes token auth verb to ray:write ( #60411 ) Replace RAY_AUTH_MODE=k8s with separate config for Kubernetes token auth ( #59621 ) Optimize token auth: use shared_ptr caching and avoid per-RPC construction ( #59500 ) Optimize OpenTelemetry metric recording calls ( #59337 ) Throttle infeasible resource warning ( #59790 ) Add default excludes for working_dir uploads ( #59566 ) Tell users why objects cannot be reconstructed ( #59625 ) Extend instance allocation timeout in autoscaler v2 ( #60392 ) Remove GCS centralized scheduling ( #59979 , #60121 , #60188 ) Demote stale sync message drop log to DEBUG in RaySyncer ( #59616 ) Migrate remaining std::unordered_map to absl::flat_hash_map ( #59921 ) Add missing fields to NodeDefinitionEvent proto ( #60314 ) Add actor and task event missing fields ( #60287 ) Add node id to the base event ( #59242 ) Add repr_name to actor_lifecycle_event ( #59925 ) Support ALL in exposable event config ( #59878 ) Support publishing events from aggregator to GCS ( #55781 ) Update the attempt number of actor creation task when actor restarts ( #58877 ) Unify node feasibility and availability checking for GPU fractions ( #59278 ) Update TPU utils for multi-slice compatibility ( #59136 ) Improve SubprocessModuleHandle.destroy_module() resource cleanup ( #60172 ) Support viewing PIDs for Dashboard and Runtime Env Agent ( #58701 ) Optimize autoscaler monitor by moving resource demand parsing outside loop ( #59190 ) Avoid GCS query for is_head in dashboard agent startup ( #59378 ) Skip reporter and event aggregator client creation in minimal mode ( #59846 ) Support out-of-order actors by extracting metadata when creating (RDT) ( #59610 ) Synchronize CUDA stream before registering for NIXL ( #60072 ) Atomically send/recv for two-sided ordering (RDT) ( #60202 ) Add get_session_name() to RuntimeContext ( #59469 ) Make MAX_APPLICATION_ERROR_LEN configurable via env var ( #59543 ) Preserve function signatures through Ray decorators ( #60479 ) 🔨 Fixes Fix idle_time_ms resetting for nodes not running tasks ( #60581 ) Fix task event loss during shutdown ( #60247 ) Filter bad subscriber messages from taking down GCS publisher ( #60252 ) Fix RAY_EXPERIMENTAL_NOSET_* environment variable parsing in accelerator managers ( #60577 ) Fix ray start --no-redirect-output crash ( #60394 ) Fix drain state propagation race condition ( #59536 ) Fix use-after-free race condition in OpenTelemetry gauge metric callback during shutdown ( #60048 ) Fix PSUTIL_PROCESS_ATTRS returning empty list on Windows ( #60173 ) Fix deadlock in garbage collection when holding lock ( #60014 ) Fix incorrect error handling in autoscaler for available_node_types on on-prem clusters ( #60184 ) Fix invalid status transitions in autoscaler v2 ( #60412 , #59550 ) Fix GCS crash from race condition in MetricsAgentClient exporter initialization ( #59611 ) Fix tracing signature mismatch when calling actors from different processes ( #59634 ) Fix crash when killing actor handle from previous session ( #59425 ) Fix multiple deployment same name resolve ( #59577 ) Handle dual task errors with read-only args ( #59507 ) Handle exceptions raised by internal_ip() within StandardAutoscaler ( #57279 ) Fix uv_runtime_env_hook.py to pin worker Python version ( #59768 ) Fix STRICT_PACK placement groups ignoring bundle label selectors ( #60170 ) Fix logging bug when log value is an empty string ( #59434 ) Fix aggregator-to-GCS event conversion ( #59783 ) Raise error on tail log job error in newer Ray versions ( #59506 ) Fix num retries left message ( #59829 ) Fix psutil internal API usage in dashboard disk usage reporting ( #59659 ) Fix event exporter init ray check ( #60073 ) Prevent use-after-free error in core worker shutdown ( #58435 ) Fix task name inconsistency in RUNNING vs FINISHED metrics ( #59893 ) Fix symmetric_run using wrong condition to check GCS readiness ( #59794 ) Preserve Pydantic details when serialization fails ( #59401 ) Retry GCP project metadata updates on HTTP 412 errors ( #60429 ) Fix v1 autoscaler TypeError when using bundle_label_selectors ( #59850 ) Shorten SHA-256 hex with base32 to comply with GCP label limits ( #60722 ) 📖 Documentation Add initial user guide for Ray resource isolation with writable cgroups ( #59051 ) Add token authentication internals documentation ( #59299 ) Update metric exporter docs ( #59874 ) Add internal documentation for Port Service Discovery ( #59844 ) Update misleading Ray job diagram ( #59940 ) Add debugging logs related to pinned argument size limit ( #60175 ) Add slow startup tip to podman troubleshooting docs ( #59942 ) Clarify ray.shutdown() behavior for local vs remote clusters ( #59845 ) Improve placement group fault tolerance doc ( #59830 ) Add head-node memory growth and OOM guidance ( #58695 ) Add documentation for RAY_RUNTIME_ENV_BEARER_TOKEN env var ( #60136 ) Dashboard 💫 Enhancements Support more panels in dashboard ( #60018 ) Add autoscaler metrics to Data Dashboard ( #60472 ) Support viewing PIDs for Dashboard and Runtime Env Agent ( #58701 ) 🔨 Fixes Update total for dark mode color ( #60106 ) Ray Wheels and Images Add experimental Python 3.13 images ( #59811 , #60549 ) Drop Python 3.9 support completely ( #59824 , #59825 , #59769 ); wheel Python version now declared as 3.10+ ( #59835 ) ray-cpp wheels are now py3-none , without specific Python versions. ( #59969 ) Documentation Add committership documentation ( #60069 ) Update contribution guide with common labels ( #59473 ) Add KubeRay & Volcano integration docs update ( #59636 ) Add RayJob InTreeAutoscaling with Kueue docs after Kueue 0.16.0 release ( #59648 ) Refactor LLM batch inference template ( #59897 ) Add async inference template ( #58393 ) Add RunLLM chat widget for Ray docs ( #59126 ) Fix various typos and broken links ( #60249 , #59901 , #60181 ) Replace Ray Tune + Train example with vanilla Ray Tune in homepage ( #60229 ) Add Ray technical charter ( #60068 ) Thanks Thank you to everyone who contributed to this release! @KaisennHu , @MiXaiLL76 , @slfan1989 , @krisselberg , @JasonLi1909 , @Priya-753 , @pseudo-rnd-thoughts , @zzchun , @ZacAttack , @pushpavanthar , @jjyao , @ryanaoleary , @pcmoritz , @akshay-anyscale , @HassamSheikh , @yurekami , @Hyunoh-Yeo , @ruoliu2 , @nrghosh , @wxwmd , @myandpr , @J-Meyers , @trilamsr , @kouroshHakha , @limarkdcunha , @manhld0206 , @jreiml , @preneond , @yuchen-ecnu , @Yicheng-Lu-llll , @AchimGaedkeLynker , @vaishdho1 , @israbbani , @OneSizeFitsQuorum , @Sathyanarayanaa-T , @nadongjun , @xinyuangui2 , @Rob12312368 , @as-jding , @lee1258561 , @popojk , @coqian , @rajeshg007 , @jeffreywang-anyscale, @kamil-kaczmarek , @alexeykudinkin , @Aydin-ab , @mgchoi239 , @dragongu , @edoakes , @smortime , @tk42 , @abrarsheikh , @jakubzimny , @Future-Outlier , @axreldable , @owenowenisme , @g199209 , @cem-anyscale , @dayshah , @akelloway , @daiping8 , @dlwh , @robertnishihara , @400Ping , @matthewdeng , @antoine-galataud , @cristianjd , @Partth101 , @goutamvenkat-anyscale , @codope , @seanlaii , @andrew-anyscale , @andrewsykim , @liulehui , @simonsays1980 , @Sparks0219 , @yifanmai , @landscapepainter , @win5923 , @kangwangamd , @srinarayan-srikanthan , @KeeProMise , @srinathk10 , @my-vegetable-has-exploded , @MengjinYan , @yancanmao , @yuhuan130 , @ArturNiederfahrenhorst , @akyang-anyscale , @rushikeshadhav , @kongjy , @harshit-anyscale , @justinvyu , @dancingactor , @Vito-Yang , @cr7258 , @marwan116 , @muyihao , @DeborahOlaboye , @bveeramani , @kriyanshii , @khluu , @machichima , @Kunchd , @jonded94, @iamjustinhsu , @sampan-s-nayak , @wingkitlee0 , @sunsetxh , @dkhachyan , @can-anyscale , @TimothySeah , @raulchen , @elliot-barn , @ryankert01 , @xyuzh , @stephanie-wang , @hao-aaron , @simeetnayan81 , @cszhu , @richardliaw , @yuanjiewei , @kyuds , @eicherseiji , @RedGrey1993 , @rueian , @jeffreyjeffreywang , @crypdick , @ankur-anyscale , @aslonnie

Highlights Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see this RFC for details. Ray Data now has support for bounded reading from Kafka and improved Iceberg support. Ray Data 🎉 New Features Autoscaling : New utilization-based cluster autoscaler for Ray Data workloads ( #59353 , #59362 , #59366 ). To use this new autoscaler set RAY_DATA_CLUSTER_AUTOSCALER=V2. Kafka Datasource : Add Kafka as a native datasource for data ingestion ( #58592 ) Dataset summary API : Add Dataset.summary() API for quick dataset inspection ( #58862 ) Iceberg support : Add Iceberg schema evolution, upsert, and overwrite support ( #59210 , #59335 ) Graceful error handling : Add should_continue_on_error for graceful error handling in batch inference ( #59212 ) Datetime compute expressions : Add datetime compute expressions support ( #58740 ) Grouped with_column expressions : Enable expressions for grouped with_column in Ray Data ( #58231 ) Parallelized collation : Parallelize DefaultCollateFn , arrow_batch_to_tensors ( #58821 ) 💫 Enhancements Optimized Autoscaler Step Size : Optimize autoscaler to support configurable step size for actor pool scaling ( #58726 ) Improved Streaming Repartition : Improve streaming repartition performance ( #58728 ) Actor init retry : Add actor retry if there's a failure in __init__ ( #59105 ) Fused Repartition + MapBatches : Fuse StreamingRepartition with MapBatches operators to scale collate ( #59108 ) Combined repartitions : Combine consecutive repartitions for efficiency ( #59145 ) Prefetch buffering : Handle prefetch buffering in iter_batches ( #58657 ) HashShuffle block breakdown : HashShuffleAggregator breaks down blocks on finalize ( #58603 ) Backpressure tuning : Tune concurrency cap backpressure object store budget ratio ( #58813 ) Non-string ApproximateTopK : Support non-string items for ApproximateTopK aggregator ( #58659 ) Lance version support : Add version support to read_lance() ( #58895 ) Dashboard metrics : Add time_to_first_batch and get_ref_bundles metrics to data dashboard ( #58912 ) Iter prefetched bytes stats : Add iter_prefetched_bytes statistics tracking ( #58900 ) Configurable batching for iter_batches : Add configurable batching for resolve_block_refs to speed up iter_batches ( #58467 ) Improved dashboard metrics : Improve Ray Data dashboard metrics display ( #58667 ) Histogram percentiles : Update Ray Data histograms to show percentiles in data dashboard ( #58650 ) Deprecated API removal : Remove deprecated read_parquet_bulk API ( #58970 ) Block shaping option : Add disable block shaping option to BlockOutputBuffer ( #58757 ) Removed concurrency lock : Remove concurrency lock for better performance ( #56798 ) 🔨 Fixes Fixes to Unique : Fix support of list types for Unique aggregator ( #58916 ) Parquet NaN fix : Fix reading from written parquet for numpy with NaNs ( #59172 ) Hash Shuffle empty block : Fix empty block sort in hash shuffle operator ( #58836 ) Hive partitioning pushdown : Fix pushdown optimizations with Hive partitioning ( #58723 ) Object Store usage reporting : Fix obj_store_mem_max_pending_output_per_task reporting ( #58864 ) Pyarrow FileSystem serialization fix : Handle filesystem serialization issue in get_parquet_dataset ( #57047 ) Azure UC SAS : Handle Azure UC user delegation SAS ( #59393 ) Async UDF Thread Cleanup : Close threads from async UDF after actor died ( #59261 ) Object Locality Default : Default return 0s for object locality instead of -1s ( #58754 ) 📖 Documentation Added contributing guide to Ray Data documentation ( #58589 ) Added download expression to key user journeys in documentation ( #59417 ) Added Kafka user guide ( #58881 ) Added unstructured data templates from Ray Summit 2025 ( #57063 ) Improved instructions for reading Hugging Face datasets ( #58492 , #58832 ) Refined batch-format guidance in docs ( #58971 ) Exposed vision_preprocess and vision_postprocess in VLM docs ( #59012 ) Added upgrading huggingface_hub instruction ( #59109 ) Added scaling out expensive collation functions doc ( #58993 ) Ray Serve 🎉 New Features Deployment topology visibility. Exposes deployment dependency graphs in Serve REST API, allowing users to visualize and understand the DAG structure of their applications. ( #58355 ) External autoscaler integration. Adds external_scaler_enabled flag to application config, enabling third-party autoscalers to control replica counts. ( #57727 , #57698 ) Node rank and local rank support. Extends replica rank system to track node-level and per-node local ranks, enabling better distributed serving coordination for multi-node deployments. ( #58477 , #58479 ) Custom batch size function. Allows users to define custom functions for computing logical batch sizes in @serve.batch , useful when batch items have varying weights (e.g., token counts in LLM inference). ( #59059 ) Stateful application-level autoscaling. Adds policy state persistence for custom autoscaling policies, allowing policies to maintain state across control-loop iterations. ( #59118 ) New autoscaling, batching, and routing metrics. Adds Prometheus metrics for autoscaling decisions ( ray_serve_deployment_target_replicas , ray_serve_autoscaling_decision_replicas ), batching statistics, and router queue latency for improved observability. ( #59220 , #59232 , #59233 ) 💫 Enhancements Smarter downscaling behavior. Prioritizes stopping most recently scaled-up replicas during downscale, preserving long-lived replicas that are optimally placed and fully warmed up. ( #52929 ) Autoscaling performance optimizations. Short-circuits metric aggregation for single time series cases (O(n log n) → O(1)) and lazily evaluates expensive autoscaling context fields to reduce controller CPU usage. ( #58962 , #58963 ) Route matching cleanup. Removes redundant route matching logic from replicas since correct route values are now included in RequestMetadata. Also allows multiple methods ( GET , PUT ) corresponding to a route. ( #58927 ) Deployment wrapper metadata preservation. Wrapper classes from decorators like @ingress now preserve original class metadata ( __qualname__ , __module__ , __doc__ , __annotations__ ). ( #58478 ) Improved type annotations. Enhances generic type annotations on DeploymentHandle , DeploymentResponse , and DeploymentResponseGenerator for better IDE support and type inference. Adds .result() stub to DeploymentResponseGenerator to fix static typing errors. ( #59363 , #58522 ) 🔨 Fixes YAML serialization for autoscaling enums. Fixes RepresenterError when using serve build with AggregationFunction enum values in autoscaling config. ( #58509 ) Autoscaling context timestamp fix. Correctly sets last_scale_up_time and last_scale_down_time on autoscaling context. ( #59057 ) Deadlock in chained deployment responses. Fixes hang when awaiting intermediate DeploymentResponse objects in a chain of deployment calls from different event loops. ( #59385 ) FastAPI class-based view inheritance. Fixes make_fastapi_class_based_view to properly handle inherited methods. ( #59410 ) 📖 Documentation Async I/O best practices guide. New documentation covering async programming patterns and best practices for Ray Serve deployments. ( #58909 ) Replica scheduling guide. New documentation covering compact scheduling, placement groups, custom resources, and guidance on when to use each feature. ( #59114 ) Ray Train 🎉 New Features Worker Placement with Label Selectors: Added label_selector to ScalingConfig . This allows users to control worker placement by targeting specific labeled nodes in the cluster. ( #58845 , #59414 ) Multihost JaxTrainer on GPU: Introduced support for JaxTrainer running on GPU machines. ( #58322 ) Checkpoint Consistency Modes: Added CheckpointConsistencyMode to get_all_reported_checkpoints , providing options for handling checkpoint retrieval consistency. ( #58271 ) Per-Dataset Execution Options: DataConfig now supports setting execution_options on a per-dataset basis for finer-grained control over data loading. ( #58717 ) 💫 Enhancements Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. ( #58537 ) Non-Blocking Checkpoint Retrieval: get_all_reported_checkpoints no longer blocks when only metrics are reported. ( #58870 ) Improved Resource Cleanup: Implemented eager cleanup of data resources and placement groups upon training run failures or aborts, preventing resource leaks. ( #58325 , #58515 ) 🔨 Fixes MLflow Compatibility: Updated setup_mlflow API to ensure full compatibility with Ray Train V2. ( #58705 ) Validation for Checkpoint Uploads: A ValueError is now raised if checkpoint_upload_fn fails to return a valid checkpoint. ( #58863 ) 📖 Documentation New API Documentation: Added comprehensive documentation for the ray.train.get_all_reported_checkpoints method. ( #58946 ) Ray Tune 💫 Enhancements: Nested Metrics Support: Result.get_best_checkpoint now supports nested metrics, allowing for more flexible metric tracking and checkpoint selection. ( #58537 ) Ray LLM 💫 Enhancements Cloud filesystem restructuring with provider-specific implementations ( #58469 ) Bump transformers to 4.57.3 ( #58980 ) Ray Data LLM config refactor ( #58298 ) Update vllm_engine.py to check for VLLM_USE_V1 attribute ( #58820 ) Infer VLLM_RAY_PER_WORKER_GPUS from fractional placement-group bundles automatically ( #58949 ) 🔨 Fixes Fix LLM DP release test configuration ( #59090 ) Ray RLlib 🎉 New Features DreamerV3: allow num_env_runners \> 0 ( #58495 ) 💫 Enhancements 🔥 MetricsLogger tweaks+ Stats rewrite ( #56838 ) move restart message into EnvRunner ( #56750 ) make “Footsies” less verbose (optionally) ( #58939 ) update an AlgorithmConfig deprecated argument with incorrect behavior/semantics ( #59138 ) Examples/docs cleanup: merge tuned examples into examples/ ( #58893 ) move old API examples ( #59159 ) move example run scripts ( #59160 ) remove Torch 2.x doc tied to removed benchmarks ( #59173 ) remove rllib/benchmark(s) folder from RLlib directory ( #59158 ) Testing / CI & infra cleanup (part of a larger effort to organize + harden RLlib testing): clean up tests folder layout in favor of /component/tests ( #58890 ) re-enable and fix nightly tests for APPO on Atari and MuJoCo ( #58853 ) re-enable all RLlib doctests ( #58974 ) add pytest reporting hook ( pytest_runtest_makereport ) across tests ( #59003 ) add/enable RLlib Py3.10 CI lane ( #59226 ) fix as-release-test silently failing ( #59386 ) fix recursive imports in old test-utils location ( #59435 ) Remove asv.conf.json ( #58934 ) Update requirement for byod_rllib.sh ( #59157 ) 🔨 Fixes Fix custom model-config mismatch between EnvRunner and Learner ( #58739 ) MultiAgentEnvRunner: prevent double-calling connectors ( #58931 ) Error handling: log or raise when a case is not fully handled ( #58889 ) Error handling: error out when data cannot be loaded ( #59002 ) Assorted RLlib bugfixes ( #59386 ) 📖 Documentation Update APPO paper reference to link to IMPACT paper ( #58935 ) Ray Core 🎉 New Features Support zero-copy serialization for read-only PyTorch tensors via RAY_ENABLE_ZERO_COPY_TORCH_TENSORS ( #57639 ) Add .rayignore file support for controlling cluster uploads ( #58500 ) Improve large-scale resource view synchronization through sync message batching ( #57641 ) Autoscaler with cloud resource availability awareness ( #58623 ) Token authentication UX improvements with new AuthenticationError exception ( #58737 ) Support X-Ray-Authorization fallback header for auth token in dashboard ( #58819 ) 💫 Enhancements Limit core worker gRPC reply threads to 2 by default via RAY_core_worker_num_server_call_thread ( #58771 ) Make accessor node address and liveliness cache thread safe ( #58947 ) Create OtlpGrpcMetricExporter wrapper to log export failures ( #58929 ) Print detailed exception information when failing to report events ( #58953 ) Simplify local/global GC logic ( #58671 ) Surface correct error message when get_if_exists=True for actor lookup ( #58628 ) Throw AuthenticationError from Python for token loading errors ( #59031 ) Use secrets.token_hex(32) to generate auth tokens ( #58818 ) Remove AUTH_MODE=token check in get-auth-token CLI ( #58848 ) Introduce core chaos network release tests ( #58868 ) 🔨 Fixes Fix grpc_authentication_server_interceptors streaming response handling ( #59104 ) Fix handle leak in IsProcessAlive on Windows ( #59106 ) Fix counter metric default branch for RAY_enable_open_telemetry ( #59095 ) Fix leaking metric recorder in tests ( #58952 ) Fix crash when using JVM HDFS by adding RAY_DISABLE_FAILURE_SIGNAL_HANDLER option ( #58984 ) Fix heap corruption in RayletClient causing driver crash (use-after-free) ( #58660 ) Use shared_ptr for pins_in_flight_ to prevent use-after-free ( #58744 ) Remove deprecated add_command_alias ( #58719 ) Remove cluster_full_of_actors_detected_* fields (unused in autoscaler v2) ( #59052 ) 📖 Documentation Add token-auth.md documentation page ( #58829 ) Update KubeRay authentication guide to use native Ray token authentication ( #58729 ) Dashboard 💫 Enhancements Add time_to_first_batch and get_ref_bundles metrics to data dashboard ( #58912 ) Update Ray Data histograms to show percentiles grouped by operator ( #58650 ) Ray Wheels and Images Upgraded rich , cupy-cuda12x , and memray ( #58983 ) Upgraded lxml to 6.0.2 ( #58808 ) Upgraded requests from 2.32.3 to 2.32.5 ( #58724 ) Added openlineage-python in the dependency set ( #58724 ) Thanks Thank you to everyone who contributed to this release! @xinyuangui2 , @harshit-anyscale , @Sparks0219 , @israbbani , @siyuanfoundation , @robertnishihara , @thomasdesr , @spencer-p , @aslonnie , @ZacAttack , @soodoshll , @marosset , @simeetnayan81 , @soffer-anyscale , @abrarsheikh , @400Ping , @richo-anyscale , @as-jding , @rueian , @kshanmol , @yancanmao , @zzchun , @coqian , @matthewdeng , @Future-Outlier , @YoussefEssDS , @ykdojo , @pseudo-rnd-thoughts , @lowdy1 , @ArturNiederfahrenhorst , @myandpr , @komikndr , @machichima , @RisinT96 , @curiosity-hyf , @alanwguo , @CaiZhanqi , @Aydin-ab , @MengjinYan , @suzuri-lollipop , @jeffreyjeffreywang , @rushikeshadhav , @alexeykudinkin , @meAmitPatil , @zcin , @teddygood , @elliot-barn , @dayshah , @srinathk10 , @XLC127 , @simonsays1980 , @kevin85421 , @bveeramani , @kunling-anyscale , @khluu , @andrew-anyscale , @KaisennHu , @kouroshHakha , @ryankert01 , @pavitrabhalla , @jjyao , @dragongu , @SolitaryThinker , @justinrmiller , @wxwmd , @Haustle-v , @TimothySeah , @goutamvenkat-anyscale , @liulehui , @raulchen , @HassamSheikh , @Priya-753 , @vaishdho1 , @dancingactor , @daiping8 , @eloaf , @JasonLi1909 , @rayci-bot , @richardliaw , @SheldonTsen , @Yicheng-Lu-llll , @ktyxx , @pschmutz , @iamjustinhsu , @ahao-anyscale, @cem-anyscale , @eicherseiji , @edoakes , @rajeshg007 , @arki05 , @andrewsykim , @nrghosh , @ryanaoleary , @kyuds , @Daraan , @can-anyscale , @sampan-s-nayak , @xyuzh , @owenowenisme

Fix for CVE-2025-62593 : reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic