6 bernhard dev by Kait0 · Pull Request #580 · Emerge-Lab/PufferDrive

Kait0 · 2026-08-28T16:21:45Z

Changes I made while the others were on vacation.

Update maps, fixes holes, add traffic light group information (can be used optionally to control TL during eval)
Update default parameters to match GIGAFLOW more closely
Fix phantom braking and match to uniform distribution in the paper
Add true FP32 support (with TF32 / 19 bit turned off)
Fix a bunch of smaller bugs around spawning
Made red light infraction be relative to the vehicle center to avoid problems with virtually making the vehicle bigger during eval
Added jerk threshold for starting to drive when standing still. Remove a bunch of long tail infractions due to low probabilites making the vehicle creep.
Move noralization more to -1, 1 window
Average logging acros GPUs.
remove constant observation
Add acceleration clipping
Visualize ego inputs as well
Disentangle gradient clipping of actor and critic if they are seperate networks
Implement adavantage filter leakage option that keep a small percent of random samples to prevent bias. Turned off by default
Add option to visualize training in HTML (removed old EGL stuff)
Better training cleanup for multinode
Add paralizable evals
Some unit tests.

https://wandb.ai/emerge_/nightly-multi-long/runs/k_scaled_0028_1000
1T run. Model gets around 200k km/infraction, which is a new best.

…eset.

…moval fractions.

env.goal_source=route \ env.max_goal_spacing=60 \

…ed on the date and I do not need to touch the script anymore.,

…n command.

… tests.

vcharraut · 2026-08-31T08:05:08Z

+  # Replay-mode agents flagged mark_as_expert are log-replayed (1) or treated like any other agent (0)
+  replay_expert_agents: 1


duplicated config

you should be able to do the same thing with

Controller used by non-SDC vehicles.

options: "static", "policy", "replay", "idm"

non_sdc_controller: policy

Controller used by non-vehicle agents. "auto" follows non_sdc_controller unless it is "idm", then it uses "replay".

options: "auto", "static", "policy", "replay", "idm"

non_vehicle_controller: auto

in replay

So I only use replay_expert_agents: 0
for nuPlan and CARLA

Claude claims the 1 branch is necessary for compatibility with WOMD, but I never used WOMD, so idk.
The controllers are category-wide, whereas this flag uses the expert/non-expert flag from the dataset
I would keep it if it doesn't bother.

I see, I quoted the wrong config then

You could add a new option to control_mode like control_all_vehicles that would actually control all valid cars, enforcing controlling the mark_as_expert ones

Ideally we don't want to create one config variable for every behavior we rather try to merge into existing one when possible

vcharraut · 2026-08-31T08:15:57Z

+        PyObject *tuple = PyTuple_New(4);
        PyTuple_SetItem(tuple, 0, agent_offsets);
        PyTuple_SetItem(tuple, 1, map_ids_list);
        PyTuple_SetItem(tuple, 2, PyLong_FromLong(env_count));
+        PyTuple_SetItem(tuple, 3, PyLong_FromLong(env_count));
        return tuple;
    }


this seems useless

vcharraut · 2026-08-31T08:16:10Z

+            PyObject *tuple = PyTuple_New(4);
            PyTuple_SetItem(tuple, 0, agent_offsets);
            PyTuple_SetItem(tuple, 1, map_ids_list);
            PyTuple_SetItem(tuple, 2, PyLong_FromLong(env_count));
+            PyTuple_SetItem(tuple, 3, PyLong_FromLong(env_count));
            return tuple;
        }


this seems useless

Slot 3 is maps_consumed, Slot 2 is num_envs. Both are used in the code. We could also set the equivalence elsewhere I guess?

vcharraut · 2026-08-31T08:18:20Z

+    // Per-episode perturbation rates: uniform in [0, configured max]
+    env->episode_partner_blindness_prob = sample_uniform(&env->rng_state, 0.0f, env->partner_blindness_prob);
+    env->episode_phantom_braking_prob = sample_uniform(&env->rng_state, 0.0f, env->phantom_braking_prob);


why do we want a probability for all agents inside an episode vs having a probality per agent ?

To match the gigaflow paper which have an up to 5% number of phantom brakers. That is the total number of phatom brakers in an episode.
The individual phantom braker still has a probability for it to trigger.
We want agents that experience clean episodes and agents that experience crazy epsiodes I think.
Having it per agent would mean that on average all scenarios look the same due to many draws?

Ok I understand. In my comprehension of the paper, the "Up to 10% of agents" refers to all the agents batch-wise, so not per episode

I have no idea which one is correct

yeah, we can ask eugene when he is back. The uniform distribution I use is also an assumption. Though I think this change did help to reduce the rear end collisions, so I would keep it unless eugene says he has a better solution.

vcharraut · 2026-08-31T08:21:46Z

+static bool walk_to_next_goal(
+    Drive *env,
+    const int *route,
+    int route_length,
+    int cursor_idx,
+    float s_on_lane,
+    float spacing_meters,
+    float prev_heading,
+    float *out_x,
+    float *out_y,
+    float *out_z,
+    int *out_lane_idx,
+    int *out_cursor_idx,
+    float *out_s_on_lane,
+    float *out_heading) {
+    // One goal step forward. In free-roam mode with goal_heading_max_deg > 0, re-samples the spacing until the
+    // landed lane heading is within that angle of prev_heading; accepts the last attempt if none qualifies.
+    bool constrain_heading = route == NULL && env->goal_heading_max_deg > 0.0f;
+    float max_heading_delta_rad = env->goal_heading_max_deg * (float) M_PI / 180.0f;
+    for (int attempt_idx = 0; attempt_idx < GOAL_HEADING_MAX_ATTEMPTS; attempt_idx++) {
+        if (attempt_idx > 0) {
+            spacing_meters = sample_uniform(&env->rng_state, env->min_goal_spacing, env->max_goal_spacing);
+        }
+        if (!route_point_at_distance(
+                env,
+                route,
+                route_length,
+                cursor_idx,
+                s_on_lane,
+                spacing_meters,
+                out_x,
+                out_y,
+                out_z,
+                out_lane_idx,
+                out_cursor_idx,
+                out_s_on_lane,
+                out_heading)) {
+            return false;
+        }
+        if (!constrain_heading) {
+            return true;
+        }
+        if (fabsf(normalize_heading(*out_heading - prev_heading)) <= max_heading_delta_rad) {
+            return true;
+        }
+    }
+    return true;
+}


do you know if without this we actually have a lot of goals that are not respecting the constraint ?

I have deliberatly skipped this constraint because I assumed the goal sampling for GIGAFLOW paper was only done by doing a random sampling of a lane point within an euclidian distance. But if we follow the lane topology, I expect that we have goals with no shard heading most of time

I just tried to match it to the gigaflow implementation. I don't think this made any major difference

I'm afraid this implementation can favorise closer goals -> the closer the goal is, the more likely it is to respect the 60 degrees contrainst

It would be preferable to move this change to a dedicated PR so we can better emphasize the impact of it

vcharraut · 2026-08-31T08:26:43Z

+static bool log_pose_overlaps_created_agent(
+    Drive *env,
+    Agent *agent,
+    const int *created_agent_indices,
+    int created_agent_count) {
+    for (int i = 0; i < created_agent_count; i++) {
+        Agent *other = &env->agents[created_agent_indices[i]];
+        float dx = other->sim_x - agent->sim_x;
+        float dy = other->sim_y - agent->sim_y;
+        float max_overlap_dist = agent->radius + other->radius;
+        if (dx * dx + dy * dy > max_overlap_dist * max_overlap_dist) {
+            continue;
+        }
+        if (fabsf(other->sim_z - agent->sim_z) > Z_BUFFER) {
+            continue;
        }
+        if (check_obb_collision(agent, other)) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static bool spawn_near_drivable_lane(Drive *env, Agent *agent) {
+    GridMapEntity entity_list[ROAD_QUERY_ENTITY_COUNT];
+    int list_size = get_neighbors_entities(
+        env,
+        agent->sim_x,
+        agent->sim_y,
+        entity_list,
+        ROAD_QUERY_ENTITY_COUNT,
+        ROAD_OFFSETS,
+        25);
+    for (int i = 0; i < list_size; i++) {
+        int entity_idx = entity_list[i].entity_idx;
+        int geometry_idx = entity_list[i].geometry_idx;
+        RoadMapElement *element = &env->road_elements[entity_idx];
+        if (!is_drivable_road_lane(element->type)) {
+            continue;
+        }
+        if (fabsf(element->z[geometry_idx] - agent->sim_z) > Z_BUFFER) {
+            continue;
+        }
+        float lane_distance = compute_point_to_segment_distance(
+            agent->sim_x,
+            agent->sim_y,
+            element->x[geometry_idx],
+            element->y[geometry_idx],
+            element->x[geometry_idx + 1],
+            element->y[geometry_idx + 1]);
+        if (lane_distance <= REPLAY_SPAWN_MAX_LANE_DISTANCE_M) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static bool replay_spawn_fit_for_control(
+    Drive *env,
+    Agent *agent,
+    const int *created_agent_indices,
+    int created_agent_count) {
+    if (agent->sim_length > REPLAY_MAX_CONTROLLED_LENGTH_M) {
+        return false;
+    }
+    if (!spawn_near_drivable_lane(env, agent)) {
+        return false;
+    }
+    if (check_spawn_offroad(env, agent, REPLAY_SPAWN_EDGE_CLEARANCE_M)) {
+        return false;
+    }
+    Agent inflated = *agent;
+    inflated.sim_length += 2.0f * REPLAY_SPAWN_LONGITUDINAL_CLEARANCE_M;
+    update_agent_radius(&inflated);
+    for (int i = 0; i < created_agent_count; i++) {
+        Agent *other = &env->agents[created_agent_indices[i]];
+        if (other == agent) {
+            continue;
+        }
+        float dx = other->sim_x - inflated.sim_x;
+        float dy = other->sim_y - inflated.sim_y;
+        float max_overlap_dist = inflated.radius + other->radius;
+        if (dx * dx + dy * dy > max_overlap_dist * max_overlap_dist) {
+            continue;
+        }
+        if (fabsf(other->sim_z - inflated.sim_z) > Z_BUFFER) {
+            continue;
+        }
+        if (check_obb_collision(&inflated, other)) {
+            return false;
+        }
+    }
+    return true;
+}


Ideally this code should be in 123Drive.
But for direct solution as it is, it would better to do inside the init function, similar to remove_bad_trajectories. Either enhance this function or append a new function after

I think spawn point selection, is somewhat user specific not map specific?
Think this is called twice throughout the code so in the init doesn't make so much sense.

vcharraut · 2026-08-31T15:29:25Z

+  # Also keep this random fraction of below-threshold transitions; 0 disables the leak
+  adv_filter_leak_fraction: 0.0


is this used in training ?

Currently no. I tried 5% leak but no conclusive improvement. Left it there because I think it is a sensible idea that I might want to revisit later.

vcharraut · 2026-08-31T15:34:56Z

  anneal_lr: true
  precision: bfloat16
+  # false: pure float32 matmuls/convs (no TensorFloat-32); requires precision float32
+  tf32: true


i think we want a better naming for this one

vcharraut · 2026-08-31T15:37:46Z

+    float center_x = agent->sim_x;
+    float center_y = agent->sim_y;
+    float prev_center_x = agent->prev_x;
+    float prev_center_y = agent->prev_y;


use directly the variable, no need for aliases

Kait0 added 30 commits August 12, 2026 18:13

Change seeding with better PRNG generator and pass seeds through on r…

d64c7e2

…eset.

Train model

8a96886

revert changes that make it more likely that the model stops.

93af737

Remove dropping partial minibatch size to keep updating with large re…

a268ef8

…moval fractions.

Train 8 seeds. Please converge again O_O

4da1a88

k_exp_0009

417f941

k_exp_0010 Try to reproduce best model.

695bd0e

k_exp_0011 test env.goal_speed=1000 \

45e8244

env.goal_source=route \ env.max_goal_spacing=60 \

Only rebuild source if the files have changed.

b6e983e

eval model.

d6c7cbc

eval k_exp_0009_2000

2a357ec

eval k_exp_0009_3000

e893249

eval k_exp_0009_7000

3d9c368

log final evals to wandb after training.

30c61cc

Fix mean action computation.

7821774

reeval k_0009 with fixed mean computation

cd7a922

make eval folder configurable. Rerun k_exp_0009_ with mode.

454dee2

Merge remote-tracking branch 'origin/3.0' into 3_bernhard_dev

56456b2

Make mean the default for evaluation

9425b68

remove mean paramter as it is the default now.

ff05958

update unit tests.

e43f9f2

Update nightly script such that it automatically changes the name bas…

e7461b2

…ed on the date and I do not need to touch the script anymore.,

Update nightly script to use the nightly PufferDrive folder.

d2c3c64

Fix script for nightly run

149fdcd

add conttab.txt

4b005bd

k_exp_0012 main + 128 rotated and translated maps.

c86fae0

small fix in nuPlan path

b592e1c

render eval of best model

09135ae

remove eval array.

bddfade

disable meta during eval upload so that it doesn't overwrite the trai…

46c5fde

…n command.

Kait0 added 25 commits August 27, 2026 17:15

5x more driving.

65ba01e

_medium2 mode + red light infractions.

a86546b

minor

ac96fcb

Add entropy annealing to training run.

4b63100

eval sample

ddd128f

k_scaled_0030 entropy annealing.

d2ab0a3

add more info to viz

3590abf

Update viz. eval k 30

484abc3

rerun with mean

37966ae

rerun without RL

c259580

eval jerk clipping.

5d85105

Try 1.0

c9ff71e

mps3=1.5

521568e

Implement full float32 training.

4839b58

eval with old TL logic.

e80acd3

k_scaled_0028_1000 with new eval hack

e316e5d

test medium 11

1665586

eval hole fixes again

e06c1e0

_long

60feee4

Make traffic_light_junction_phases optional and update maps, fix unit…

4a92797

… tests.

update default params, fix unit test, format changes.

f8718cb

Merge remote-tracking branch 'origin/3.0' into 6_bernhard_dev

f6557ae

remove unbiased=False that snuck in

bde1b57

update golden

45a9495

precommit

f2c1158

vcharraut reviewed Aug 31, 2026

View reviewed changes

pr changes

956918a

vcharraut reviewed Aug 31, 2026

View reviewed changes

		# Replay-mode agents flagged mark_as_expert are log-replayed (1) or treated like any other agent (0)
		replay_expert_agents: 1

		# Also keep this random fraction of below-threshold transitions; 0 disables the leak
		adv_filter_leak_fraction: 0.0

Uh oh!

Conversation

Kait0 commented Aug 28, 2026

Uh oh!

Choose a reason for hiding this comment

Controller used by non-SDC vehicles.

options: "static", "policy", "replay", "idm"

Controller used by non-vehicle agents. "auto" follows non_sdc_controller unless it is "idm", then it uses "replay".

options: "auto", "static", "policy", "replay", "idm"

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vcharraut Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vcharraut Aug 31, 2026 •

edited

Loading