LLM's will be treated much like occupations for people going forward.
Just like our own education, there will be a basic set of skills all LLM's need to be trained on to provide a minimal useable product. With GPT4 I suspect we are at this point now if not slightly beyond the optimal basic training stage.
However, now it is time to send our LLM's off to university so to speak, to specialise them in certain tasks and knowledge base. The current approach of attempting to just build everything in to one does not work for a human mind, so why would it work for a vector based replication?
I suspect in the coming months we will hear more about tiny models trained on much smaller datasets and then specialised using a mix of adaptors and LoRA modifications to excel at specific tasks like code generation, translation, and conversation. Then multiple models will be implemented in one application chain to best leverage each of the respective strengths.
I suspect in the coming months we will hear more about tiny models trained on much smaller datasets and then specialised using a mix of adaptors and LoRA modifications to excel at specific tasks like code generation, translation, and conversation. Then multiple models will be implemented in one application chain to best leverage each of the respective strengths.