Summary

  1. Steering LLMs towards Code Generation depends on the Programmer providing the relevant context to the Model, for understanding the Program’s Design Principles, System Understanding and Domain Understanding.
  2. Debugging and generating new code in physical control systems is extremely difficult. It’s not fix patterns that you can rehash like in constructing certain types of games or UI.
  3. Formulating Plan for Code Improvement” can generally works well at a high (and not very useful level).
  4. Logic errors can be due to wrong high level plan, wrong implementation logic, wrong hyperparameters, or wrong input assumptions. Neither the code agent or Human has vision over what is happening.

  5. Code Generation with Agentic self-corrective Feedback, and human feedback has irrecoverable failure.

  6. Vision LMs can Providing Feedback (Labels) on the correctness of small functions, but we need to address the latency of Vision LMs for synthetic generation.


The full Deck is here