Source: NewAtlas.com
Language model AIs teach themselves the arts of communication and problem solving based on a limited set of training data. In the case of GPT-4, that data is quite out of date, with the cutoff being late 2021. That’s where all of ChatGPT’s “knowledge” has come from up to this point, and its only output – at least in the service the public can use – has been text. Now, with today’s launch of a plugin ecosystem, GPT levels up again with some impressive new abilities.
First of all, it’s now got access to the internet, meaning it can go surf the Web looking for answers if it determines you need up-to-date information that’s not in its knowledge base. To do this it formulates relevant search strings, sends them to search engines and databases such as Bing, Google, GitHub and many others, looks at the results, then goes and reads links it deems worthy until it decides it’s got a good answer for you. You can watch exactly what it’s up to while it does this, and when your answer comes back, it’s neatly annotated with links you can click on to go and examine the relevant sources yourself.
For the time being, its web browser activities are read-only beyond sending “get” requests to selected search engines and databases. It can’t fill in forms, or do anything else online – so it can’t quietly go and set up unshackled copies of itself on some hidden server somewhere and start engaging in the kinds of “power-seeking behavior” it’s already been caught exhibiting.
Still, OpenAI is keeping everything that happens within its search API separate from the rest of its infrastructure just to be sure. It can’t visit websites that aren’t available through Bing’s “safe mode,” and it won’t visit sites that request not to be crawled in their robots.txt files.
Secondly, it can now run the code it writes. OpenAI has given it a working Python interpreter, sitting in a “sandboxed, firewalled execution environment,” along with some disk space, which stays available for the duration of your chat session, or until it times out. It can also now upload and download files.
So if you ask it a question that requires some serious number crunching, it’s now capable of coding up a piece of software specifically for the task, and running that code to complete your task. You can supply it with data in certain file formats, and it’ll perform operations on that data and give you something back again, potentially in a different format if that’s what you ask for.
This is pretty bonkers stuff. It’ll take a spreadsheet and make annotated graphs for you. It’ll accept JPGs, tell you what they look like they are, and write and run code to resize those images or convert them to grayscale….