I think your job in your current form is likely in danger.

SOTA Foundation Models like GPT4 and Gemini Ultra can write code, execute, and debug with special chain of thought prompting techniques, and large acale process verification on synthetic data and RL search for correct outputs will make this 10x better. The silver lining to this is that I expect this to require an absolute shit ton of compute to constantly generate LLM output hundreds of times for each internal prompt over multiple prompts, requiring immense compute and possibly taking longer than an ordinary software engineer to run. I suspect early full stack developer LLMs will mainly be used to do a few very tedious coding tasks and SWEs will be cheaper for a fair length of time.

I expect it will be 2-3 years before this happens, so for that short period I expect workers to be “super-productive” by using LLMs in the coding process, but I expect the crossover point when the LLM becomes better is quite soon, perhaps in the next 5 years as compute requirements go down.

NFTs are stupid AF for most of the tasks people currently use them for and definitely shouldn’t be used as proof of ownership of physical assets.

However, I think NFTs make a lot of sense as proof of ownership of purely digital assets, especially those which are scarce.

For example, there are several projects for domain name resolution based on NFT ownership (e.g you look up crypto.eth, your browser checks that the site is signed by the owner of the crypto.eth NFT, then you are connected to the site), as it could replace our current system, which has literally 7 guys that hold a private key that is the backbone of the DNS system and a bunch of registrars you have to go through to get a domain. This won’t happen anytime soon but it is an interesting concept.

Then I think an NFT would also be good as a decentralized alternative to something like Google sign in, where you sign up for something with the NFT and sign in by proving your ownership of it.

In general though I find NFTs to be a precarious concept. I mean the experience I’ve had with crypto is you literally have a seed phrase for your wallet, and if it gets stolen all your funds are drained. And then for an NFT, if you click on the wrong smart contract, all your monkeys could be gone in an instant. There is in general no legal recourse to reverse crypto transactions, and I think that is frankly the biggest issue with the technology as it stands today.

Hello, kids! Pirates are very bad! Never use qBittorent to download copyrighted material, and certainly do NOT connect it to a VPN to avoid getting caught. Additionally, you should also NEVER download illegal material via an https connection because it is fully encrypted and you won’t get caught!

deleted by creator

Sam Altman: We are moving our headquarters to Japan

I think this is downplaying what LLMs do. Yeah, they are not the best at doing things in general, but the fact that they were able to learn the structure and semantic context of language is quite impressive, even if it doesn’t know what the words converted into tokens actually mean. I suspect that we will be able to use LLMs as one part of a full digital “brain”, with some model similar to our own prefrontal cortex calling the LLM (and other things like vision model, sound model, etc.) and using its output to reason about a certain task and take an action. That’s where I think the hype will be validated, is when you put all these parts we’ve been working on together and Frankenstein a new and actually intelligent system.

For the love of God please stop posting the same story about AI model collapse. This paper has been out since May, been discussed multiple times, and the scenario it presents is highly unrealistic.

Training on the whole internet is known to produce shit model output, requiring humans to produce their own high quality datasets to feed to these models to yield high quality results. That is why we have techniques like fine-tuning, LoRAs and RLHF as well as countless datasets to feed to models.

Yes, if a model for some reason was trained on the internet for several iterations, it would collapse and produce garbage. But the current frontier approach for datasets is for LLMs (e.g. GPT4) to produce high quality datasets and for new LLMs to train on that. This has been shown to work with Phi-1 (really good at writing Python code, trained on high quality textbook level content and GPT3.5) and Orca/OpenOrca (GPT-3.5 level model trained on millions of examples from GPT4 and GPT-3.5). Additionally, GPT4 has itself likely been trained on synthetic data and future iterations will train on more and more.

Notably, by selecting a narrow range of outputs, instead of the whole range, we are able to avoid model collapse and in fact produce even better outputs.

We have no moat and neither does OpenAI is the leaked document you’re talking about

It’s a pretty interesting read. Time will tell if it’s right, but given the speed of advancements that can be stacked on top of each other that I’m seeing in the open source community, I think it could be right. If open source figured out scalable distributed training I think it’s Joever for AI companies.

I definitely agree. The vast majority of people still left on Reddit are those who are corporate bootlickers and those who do not care and just want to doom scroll.

Neither type adds anything to an online community

This isn’t an actual problem. Can you train on post-ChatGPT internet text? No, but you can train on the pre-ChatGPT common crawls, the millions of conversations people have with the models and on audio, video and images. As we improve training techniques and model architectures, we will need even less of this data to train even more performant models.