• 3 Posts
  • 122 Comments
Joined 1 year ago
cake
Cake day: July 1st, 2023

help-circle

  • You don’t need to wonder, Apple has said as much that their AI is built on LLMs, just like everybody else. While hallucinations are still a major unsolved problem, that doesn’t mean they aren’t able to be reduced in frequency and severity. A ChatGPT like chatbot is going to hallucinate because you’re asking it to give extremely open ended responses to literally any query. The more data you feed it in the prompt, and the more you constrain its output, the less likely it is to hallucinate. It’ll likely be extremely rare that using the grammar check or rephrasing tools in Apple AI will be affected by hallucinations for that reason. Siri is more comparable to ChatGPT with regards to open ended questions, but it’s likely that they will integrate LLMs primarily for transforming inputs and outputs rather than the whole process. For example, the LLM could be prompted to call a function based on the user’s query. Then, that function finds a reliable result, either using existing APIs for real time information like weather, or using another LLM with a search engine. The output from this truth-finding process is then fed back into an LLM to generate the final output. The role of the LLM is heavily constrained at every step of the way, which is known to minimize hallucinations.

    You arguing that this is an unsolvable problem is defeatist and not helpful to actually mitigating the real issue.



  • As a person who has been managing Linux servers for about a decade now, trust me that a few hours or days of learning docker now will save you weeks if not months in the future. Docker makes managing servers and dealing with updates trivial and predictable. Setting everything up in docker compose makes it easy to recover if something fails, it’s it’s self documenting because you can quickly see exactly how your applications are configured and running.
















  • Is the distrust in the quality of the output? If so, I think the main thing Apple has going for it is that they use many fine tuned models for context constrained tasks. ChatGPT can be arbitrarily prompted and is expected to give good output for everything, sometimes long output. Being able to do that is… hard. However, most of apple’s applications are much, much narrower. Like, the writing assistant which will rephrase at most a few paragraphs: the output is relatively short, and the model has to do exactly one task. Or in Siri: the model has to take a command, and then select one or more intents to call. It’s likely that choosing which intents to call, and what kinds of arguments to provide are handled by separate models optimized for each case. Despite all that, it is very possible that errors can still occur, but there are fewer chances for them to occur. I think part of Apple’s motivation for partnering with OpenAI specifically for certain complex Siri questions, is that this is an area they aren’t comfortable putting Apple branding on due to output quality concerns, and by providing it with a partner, they can pass blame onto the partner. Someday if LLMs are better understood and their output can be better controlled and verified for open ended questions, that’s when Apple might dump OpenAI and advertise their in house replacement as being accurate and reliable in a way ChatGPT isn’t.


  • The privacy and security issues with LLMs are mitigated by the majority of it being on-device. Anything on device, in my opinion, has zero privacy or security issues. Anything taking place on a server has a potential to be a privacy issue, but Apple seems to be taking extraordinary measures to ensure privacy with their own systems, and ChatGPT, which doesn’t have the same protections, will be strictly opt in separately from Apple’s service. I see this as basically the best of all options, maximizing privacy while retaining more complex functionality.