I would guess that it would be unlikely for any ICC case to have significant standing in any American courthouse. The US is no longer a signatory to the Rome Statute which was the treaty which established the ICC and I don’t believe there is any interest in being bound to their rulings.
>The Code Llama models provide stable generations with up to 100,000 tokens of context.
Not a bad context window, but makes me wonder how embedded code models would pick that context when dealing with a codebase larger than 100K tokens.
And this makes me further wonder if, when coding with such a tool (or at least a knowledge that they’re becoming more widely used and leaned on), are there some new considerations that we should be applying (or at least starting to think about) when programming? Perhaps having more or fewer comments, perhaps more terse and less readable code that would consume fewer tokens, perhaps different file structures, or even more deliberate naming conventions (like Hungarian notation but for code models) to facilitate searching or token pattern matching of some kind. Ultimately, in what ways could (or should) we adapt to make the most of these tools?
You can, I suppose, contract your code so that it’s context free and uses less tokens, but that makes it more confusing for humans and language models.
Taken to the extreme, you can see obviously with one letter functions and variables like i, j, k the model will be able to infer literally nothing and, thus, produce arbitrary nonsense.
Clearly the solution is to do what we already do to manage complexity which is to decompose large tasks into smaller black box modules with an api where the (large number of tokens) implementation is hidden and not known or relevant to using it.
If you give an LLM a function signature and good description, maybe some usage examples, it doesn’t need the implementation to use it.
Terseness decreases the ability of LLMs to process code; it doesn’t solve context length, and even at best it doesn’t scale.
The process of decomposing the task into smaller steps and generate each step independently seems to be the correct way in my experience too. It works very well with GPT (chatGPT or GPT4).
>100k tokens is plenty.
The context window can be really helpful, in case there is a release of a new library and the user wants to generate code targeting the API of the library. When the training date stops at August 2023, any library released after that date is not known to the engine.
My general opinion in regards to context window, is that 1 trillion tokens context window still may not be enough for all use cases.
Your developer tool already maps out the entire code base in useful ways, such as knowing all the symbols available in the current context and the structure of classes. This information can be distilled for presentation to the LLM. For instance, if you’re wanting to generate a method implementation inside a C++ class, the LLM can be given a condensed version of the header files that the compiler would have access to on compiling that specific class. Removing white space and comments and boiling macros down saves a lot of tokens.
You can also probably skip including standard library headers since those will be well known to the LLM through its fine tuning.
Either way, consider that a typical preprocessed C++ file would push against the 100K limit even with some optimizations. You will definitely want to have some middleware doing additional refinement before presenting that file to the LLM.
I’ve found the utility of the coding LLMs gets a lot higher when you’ve got code comments and descriptive variable and function names - the LLM makes better inferences and suggestions. We’ve seen similar on data - properly tagged data and descriptive field names helps the LLM to produce much more useful responses. I’m secretly hoping the spread of these tools will finally lead my fellow developers to comment their code and stop using three character variable names.
This is my ultimate (short term) AI fear - letting it get into a feedback loop with itself, leading to perverse and incorrect results.
To state my position more clearly: I don’t think an AI could comment code from scratch very well - how would it know all the decisions made, business logic considerations, historical conventions, micro-industry standards, etc?
A good benchmark I was told once was “if a human expert couldn’t do it, an AI probably can’t either”. And commenting code I didn’t write would certainly test the bounds of my abilities
I built a VS code extension a while back that I still use that wraps GPT-4 and writes code directly in my editor.
The method I used to choose which files to feed GPT-4 was embeddings-based. I got an embedding for each file and then an embedding from the instruction + some simple processing to pick the files more likely to be relevant. It isn't perfect but good enough most of the time in medium-sized codebases (not very large ones).
The one thing I started doing because of how I implemented this is make files shorter and move stuff into different files. Having a 1k+ LOC file is prohibitive because it eats up all the context window (although with 100k context window maybe less so). I think it's a good idea to keep files short anyways.
There's other smarter things that can be done (like embed and pass individual functions/classes instead of entire files) so I have no doubt someone will build something smarter soon. You'll likely not have to change your coding patterns at all to make use of AI.
This sounds like a job for middleware. Condensing split code into a single huge file, shortening comments, removing whitespace and such can be done by a preprocessor for the llm.
You dont even need an llm for trimming whitespace, just a smart parser with language rules like ide code checkers already use. Existing llms are fine at summarizing comments, especially with language specific grammar constraints.
A good practice is to have a prompt file where you keep the information you want the model to have at its disposal. Then you put it in the start of your conversations with GPT-4. It's also good documentation for people.
You start a project by defining the task. Then as you iterate, you can add new information to the prompt. But it can be also partially automated - the model can have a view of the file structure, classes, routes, assets and latest errors.
I was really hoping that the one year update of Codex would be that - a LLM that can see deep into the project, not just code, but runtime execution, debugging, inspecting and monitoring. Something that can iterate like autoGPT. Unfortunately it didn't improve much and has weird conflicts with the native code completion in VSCode, you get freezes or doubled brackets.
Solutions exist that feed LLMS ctags, and seem to work well. The function signatures and symbols names for a code base are much smaller than the actual code.
A relatively comprehensive look into PFAS can be gained by reading the Draft state of per- and polyfluoroalkyl substances (PFAS) report published by Health Canada. [1] The Existing Substances Risk Assessment Bureau at Health Canada (which assesses risks of chemicals that are in commerce in Canada), is increasingly moving towards new analysis techniques broadly characterized as New Approach Methodologies (NAMs) (see section 6.2.5 and 7.6 in the report). So, to my understanding, part of the apparent gap in research is being plugged by the application of models which infer chemicals properties, interactions, and effects without needing to study each of them individually to gain at least an initial idea of how they might harm organisms and environments.
See:
> The evolving landscape of chemical production has rendered toxicological testing using traditional models (i.e., live animals) impractical, and advances in science coupled with ethical concerns have resulted in government agencies, including the United States (US EPA 2021b), European Union, and Canada (Bhuller et al. 2021), committing to reduce, refine, and potentially eliminate the use of mammalian models from certain regulatory testing requirements, where scientifically justified. New approach methodologies (NAMs) are broadly described by the international risk assessment community as any technology, method, approach, or a
combination of these that can be used to reduce, refine, or replace animal testing and allow more rapid and effective screening of chemicals. These methods may include the use of computer models or assays with biological molecules, cells, tissues, or organs as well as exposure measurement approaches. [1]
Location: Canada
Remote: Yes or hybrid or onsite
Willing to relocate: Yes
Technologies: Python, C++, MATLAB, Java, PostgreSQL, HTML, CSS, AWS, Django.
Résumé/CV: martyrudolf.com/resume/
Email: martywrudolf+hn at gmail dot com
Newly graduated Computer Engineer with specialization in software. 2 years of professional programming experience including positions in government, industry, and independent consulting work.
FYI, Had trouble getting to your resume. It appears the link you gave is missing the "www." and there is no auto-redirect from "martyrudolf.com" to "www.martyrudolf.com"
It's been my view that everything that one chooses to write is generally chosen from the space of all other possible things that could be said. Particularly, I think making the first exposure to your article a clickbait headline says some important things about the contents of the article and the approach of the author. Those are important signals that help those with a keen eye to weed out low signal-to-noise ratio articles.
On the flip side, there seems to be a lot of noise lately and it would be nice to have a break from it, so I'd almost want to be spared from being served these articles entirely. But then that's taking editorial powers to another level.
Location: Edmonton, Alberta, Canada
Remote: Yes or hybrid or onsite
Willing to relocate: Yes
Technologies: Python, C++, MATLAB, Java, PostgreSQL, HTML, CSS, AWS, Django.
Résumé/CV: martyrudolf.com/resume/
Email: martywrudolf+hn at gmail dot com
Newly graduated Computer Engineer with specialization in software. 2 years of professional programming experience including positions in government, industry, and independent consulting work.
