Ciro Santilli
OurBigBook.comGiven enough computational power per dollar, AGI is inevitable, but it is not sure certain ever happen given the end of end of Moore's Law.
Alternatively, it could also be achieved genetically modified biological brains + brain in a vat.
Imagine a brain the size of a building, perfectly engineered to solve certain engineering problems, and giving hints to human operators + taking feedback from cameras and audio attached to the operators.
This likely implies transhumanism, and mind uploading.
Ciro Santilli joined the silicon industry at one point to help increase our computational capacity and reach AGI.
Ciro believes that the easiest route to full AI, if any, could involve Ciro's 2D reinforcement learning games.
Ciro Santilli has felt that perhaps what is missing in 2020's AGI research is:
- the interface between:The key question is somewhat how to extract symbols out of the space-time continuous experiences.
- the continuous/noisy level (now well developed under artificial neural network techniques of the 2010's)
- and symbolic AI level AI
- more specialized accelerators that somehow interface with more generic artificial neural networks. Notably some kind of speialized processing of spacial elements is obviously hardcoded into the brain, see e.g. Section "Grid cell"
Forcing these boundaries to be tested was one of the main design goals of Ciro's 2D reinforcement learning games.
In those games, for example:Therefore, those continuous objects would also have "magic" effects that could not be explained by "simple" "what is touching what" ideas.
- when you press a button here, a door opens somewhere far away
- when you touch certain types of objects, a chemical reaction may happen, but not other types of objects
Bibliography:
This point is beautifully argued in lots of different sources, and is clearly a pillar of AGI.
Perhaps one may argue that our deep learning layers do form some kind of hierarchy, e.g. this is very clear in certain models such as convolutional neural network. But many of those models cannot have arbitrarily deep hierarchies, which appears to be a fundamental aspect of intelligence.
How to Create a Mind:
The lists of steps in my mind are organized in hierarchies. I follow a routine procedure before going to sleep. The first step is to brush my teeth. But this action is in turn broken into a smaller series of steps, the first of which is to put toothpaste on the toothbrush. That step in turn is made up of yet smaller steps, such as finding the toothpaste, removing the cap, and so on. The step of finding the toothpaste also has steps, the first of which is to open the bathroom cabinet. That step in turn requires steps, the first of which is to grab the outside of the cabinet door. This nesting actually continues down to a very fine grain of movements, so that there are literally thousands of little actions constituting my nighttime routine. Although I may have difficulty remembering details of a walk I took just a few hours ago, I have no difficulty recalling all of these many steps in preparing for bed - so much so that I am able to think about other things while I go through these procedures. It is important to point out that this list is not stored as one long list of thousands of steps - rather, each of our routine procedures is remembered as an elaborate hierarchy of nested activities.
Human Compatible: TODO get exact quote. It was something along: life goal: save world from hunger. Subgoal: apply for some grant. Sub-sub-goal: eat, sleep, take shower. Sub-sub-sub-goal: move muscles to get me to table and open a can.
From Machine Learning to Autonomous Intelligence by Yann LeCun (2023)
Source. After a bunch of B.S., LeCun goes on to describe his AGI architecture. Nothing ground breaking, but not bad either. Tagged
This section is about ideas that are thought to be part of an AGI system.
My Job is to Open and Close Doors by Mattias Pilhede (2019)
Source. An interesting humorous short meditation on common sense.Term invented by Ciro Santilli, similar to "nuclear blues", and used to describe the feeling that every little shitty job you are doing (that does not considerably help achieving AGI) is completely pointless given that we are likely close to AGI as of 2023.
Due to the failures of earlier generations, which believed that would quickly achieve AGI, leading to the AI winters, 21st researchers have been very afraid of even trying it, rather going only for smaller subste problems like better neural network designs, at the risk of being considered a crank.
While there is fundamental value in such subset problems, the general view to the final goal is also very important, we will likely never reach AI without it.
This is voiced for example in Superintelligence by Nick Bostrom (2014) section "Opinions about the future of machine intelligence" which in turn quotes Nils Nilsson:
There may, however, be a residual cultural effect on the AI community of its earlier history that makes many mainstream researchers reluctant to align themselves with over-grand ambition. Thus Nils Nilsson, one of the old-timers in the field, complains that his present-day colleagues lack the boldness of spirit that propelled the pioneers of his own generation:Nilsson’s sentiment has been echoed by several others of the founders, including Marvin Minsky, John McCarthy, and Patrick Winston.Concern for "respectability" has had, I think, a stultifying effect on some AI researchers. I hear them saying things like, "AI used to be criticized for its flossiness. Now that we have made solid progress, let us not risk losing our respectability." One result of this conservatism has been increased concentration on "weak AI" - the variety devoted to providing aids to human
thought - and away from "strong AI" - the variety that attempts to mechanize human-level intelligence
Don't be a pussy, AI researchers!!!
It is hard to overstate how low the level of this conference seems to be at first sight. Truly sad.
- www.quora.com/What-are-some-good-research-schools-PhD-for-Artificial-General-Intelligence-not-Machine-Learning/answer/Ciro-Santilli What are some good research schools (PhD) for Artificial General Intelligence (not Machine Learning)?
- 2020 towardsdatascience.com/four-ai-companies-on-the-bleeding-edge-of-artificial-general-intelligence-b17227a0b64a Top 4 AI companies leading in the race towards Artificial General Intelligence
- Douglas Hofstadter according to www.theatlantic.com/magazine/archive/2013/11/the-man-who-would-teach-machines-to-think/309529/ The Man Who Would Teach Machines to Think (2013) by James Somers
- Pei Wang from Temple University: cis.temple.edu/~wangp/
- www.reddit.com/r/agi/comments/zzfwww/are_there_people_actually_working_to_make_an_agi/
- Sergey Brin explicit internal memo aiming at AGI: techcrunch.com/2025/02/28/sergey-brin-says-rto-is-key-to-google-winning-the-agi-race/
Tagged
www.giotto.ai/Their website doesn't clearly explain their technology as of 2025.
At Giotto.ai, our technology is designed to bridge the gap between current AI capabilities and the promise of Artificial General Intelligence (AGI).
They claim to have done some work on ARC-AGI which is cool, but no clear references to what they did or if there's anything public about it.
kyutai.org/ just says:
Our mission is to build and democratize artificial general intelligence through open science
They are not-for-profit and had massive investments: techcrunch.com/2023/11/17/kyutai-is-an-french-ai-research-lab-with-a-330-million-budget-that-will-make-everything-open-source/
they also don't say at all what they are looking into for AGI, the only public thing they have are speech to speech and speech-to-text so how's that related to agi at.
raised $1b at $5b valuation on september 2024, then $2b at $30b on march 2025. lol!
From their website:
Superintelligence is within reach.
Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures.
By the rich founder of Mt. Gox and Ripple, Jed McCaleb.
Obelisk is the Artificial General Intelligence laboratory at Astera. We are focused on the following problems: How does an agent continuously adapt to a changing environment and incorporate new information? In a complicated stochastic environment with sparse rewards, how does an agent associate rewards with the correct set of actions that led to those rewards? How does higher level planning arise?
These are research institutes usually funded by rich tech bros, sometimes cryptocurrency magnates, but not necessarily.
Tagged
Tagged
Interesting dude, with some interest overlaps with Ciro Santilli, like quantum computing:
It is a bit hard to decide if those people are serious or not. Sometimes it feels scammy, but sometimes it feels fun and right!
Particularly concerning is the fact that they are not a not-for-profit entity, and it is hard to understand how they might make money.
Charles Simon, the founder, is pretty focused in how natural neurons work vs artificial neural network models. He has some good explanations of that, and one major focus of the project is their semi open source spiking neuron simulator BrainSimII. While Ciro Santilli believes that there might be insight in that, he also has doubts if certain modules of the brain wouldn't be more suitable coded directly in regular programming languages with greater ease and performance.
FutureAI appears to be Charles' retirement for fun project, he is likely independently wealthy. Well done.
- www.aitimejournal.com/interview-with-charles-simon-ceo-and-founder-futureai
- 2022 raised 2 million USD:
Creativity and AGI by Charles Simon's at AGI-22 (2022)
Source. Sounds OK!- youtu.be/ivbGbSx0K8k?t=856 general structure of the human brain 86B total, matching number of neurons in the human brain, with:
- 14B: brainstem
- 16B: neocortex
- 56B: cerebelum
- www.youtube.com/watch?t=1433 some sequencing ideas/conjectures
Machine Learning Is Not Like Your Brain by Future AI (2022)
Source. Contains some BrainSimII demos.The video from futureai.guru/technologies/brian-simulator-ii-open-source-agi-toolkit/ shows a demo of the possibly non open source version. They have a GUI neuron viewer and editor, which is kind of cool.
Machine Learning Is Not Like Your Brain by Charles Simon (2022)
Source. Not having a manipulator claw is a major issue with this one.
But they also have a co-simulation focus, which is a bit of a win.
Basically it looks like the dude got enough money after selling some companies, and now he's doing cooler stuff without much need of money. Not bad.
Marek Rosa's play thing.
AI Game - LLM-driven NPCs that can talk by Marek Rosa (2023)
Source. Not the most amazing demo, but the idea is there. Seems to be a preview for AI People. The previous working title seems to have been AI Odyssey.ndea.com/Cool. Founders are also very interested in ARC-AGI.
We believe program synthesis holds the key to unlocking AGI.
Homepage: www.numenta.com/
www.reddit.com/r/artificial/comments/b38hbk/what_do_my_fellow_ai_researchers_think_of_ben/ What do my fellow AI researchers think of Ben Goertzel and his research?
Term invented by Ciro Santilli to refer to problems that can only be solved once we have AGI.
It is somewhat of a flawed analogy to NP-complete.
This one goes all in the following themes:
- few examples to learn from. You have to carefully inspect the input examples to deduce the output rules. Rules can require specific It application ordering, so you actually generate an algorithm. It tends to be easy for humans, but sometimes not so easy!
- extensive use of geometric concepts, notably "contained inside", "adjacent to", "connected"
Bibliography:
- www.reddit.com/r/mlscaling/comments/1ht4emi/anyone_else_suspect_arcagi_was_never_much_of_a/ Anyone else suspect ARC-AGI was never much of a test of anything? (2025)
These section lists common visual primitives that a solver must first extract in order to infer solutions.
Some of these have a lot of prior world content, others less.
Many people have come up with the same idea on the Discord. Some nicely call it DSL.
Implementations:
If a color is inferred to be a background color, it contains no information and should be ignored.
Most problems tend to use black as a background color, but not all of them.
An "object" is a set of points that is understood to be one singular entity.
Contiguity and having the same color are strong indicators that something should be understood as an object.
A rectangular container.
The toplevel viewport is always implicitly understood as a special box.
There are two or more boxes drawn inside the toplevel and sharing boundaries with toplevel.
There are two toplevel boxes, one contains only input, and all output goes to the second one. The second one may also contain some input.
A path is something you obtain by somehow drawing from one point to another, e.g. a line, and then starting another drawing between two points from the end point.
Distance = 0.
Rectangle is like a box but always fully filled.
A point is a 1-square.
A dotted line is a generalized line that cycles between a color pattern, e.g.:would be a line:An extra color "transparent" may also be added to not change for that pixel.
r r g
r r g r r g r r g
A dotted path that is also a dotted line.
There is no unique solution, we just have to optimize something, often the least changed colors.
To the left of the vertical red line, count the number of each color on each row.
Then to the right, on each line draw one square of each color to the left every n columns, starting with a square on the first column to the right of the red line, where n is the count of that color.
Start with the color furthest away from the red line, and then color with colors nearer to the red line. If there's overlap, replace the old color with the new one.
Output:
- draw dotted lines
Input primitives:
- background color
- squares
- squares with color inside
- points
Transformations primitives:
- line drawing
Solution: move pieces to fill the gap on the fat object that crosses the screen. Place objects either on fat object or on other objects placed on the fat object. Anything you add must end in a rectangle.
The rules for this one are not entirely clear with the number of examples.
Also clearly if the goal is to make rectangular towers, then this is an NP-hard optimization problem in general.
Input primitives:
- same color chunk. Properties: crosses screen.
Transformation primitives:
- move solid around
- fills the gap
This existed earlier: x.com/GianpaoloGalli/status/1846144236900827413
Solution: vs are guns that shoot diagonal line of their color, when line touches another object, change line color to match that of the object, then bounce on the object and continue going with the new color
Input primitives:
- diagonal line
Assumptions:
- line don't cross each other, it is unclear how to resolve that case
Transformation primitives:
- draw line
- draw line and bounce
Input primitive:
Transformation primitives:
- draw lines
TODO I can't solve that one.
Input:
- background color
- boxes
- points inside boxes
- distance between point and box
Input: three or more containers:
- one touching top left corner
- inside it there are three monocolor objects
- one touching bottom right corner of toplevel box
- inside it there is one monocolor object
- outside of those, touching the left toplevel box edge, there is one or more point
Output:
- draw dotted path of perpendicular line
- the path color pattern comes from the color of top left objects, ordered from nearest to furthest from top le
That's Ciro Santilli's favorite. Of course, there is a huge difference between physical and non physical jobs. But one could start with replacing desk jobs!
AGI-complete in general? Obviously. But still, a lot can be done. See e.g.:
- The Busy Beaver Challenge deciders
deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/
AI achieves silver-medal standard solving International Mathematical Olympiad problems
"Autoformalization" refers to automatically converting a traditional human readable mathematical proof to a formal proof.
This section is about benchmarks designed to test mathematical reasoning.
Paper: arxiv.org/abs/2411.04872
arstechnica.com/ai/2024/11/new-secret-math-benchmark-stumps-ai-models-and-phds-alike/ mentions what the official website is unable to clearly state out:So yeah, fuck off.
The design of FrontierMath differs from many existing AI benchmarks because the problem set remains private and unpublished to prevent data contamination
The expected answer output for all problems is just one single, possibly ridiculously large, integer, which is kind of a cool approach. Similar to Project Euler in that aspect.
The most interesting aspect of this benchmark is the difficulty. Mathematical olympiad coach Evan Chen comments:[ref]
Problems in [the International Mathematical Olympiad] typically require creative insight while avoiding complex implementation and specialized knowledge [but for FrontierMath] they keep the first requirement, but outright invert the second and third requirement
Tagged
Original paper: Section "GAN paper".
The GAN paper itself does a bit of this, cool hello world:
Generative adversarial network illustrates well AI brittleness. The input looks obvious for a human, but gets completely misclassified by a deep learning agent.
This is going to be the most important application of generative AI. Especially if we ever achieve good text-to-video.
Image generators plus human ranking:
- pornpen.ai/ a bit too restrictive. Girl laying down. Girl sitting. Penis or no penis. But realtively good at it
- civitai.tv/. How to reach it: civitai.tv/tag/nun/2/
www.pornhub.com/view_video.php?viewkey=ph63c71351edece: Heavenly Bodies Part 1: Sister's Mary First Act. Pornhub title: "AI generated Hentai Story: Sexy Nun alternative World(Isekai) Stable Diffusion" Interesting concept, slide-narrated over visual novel. The question is how they managed to keep face consistency across images.
Very useful for idiotic websites that require real photos!
- thispersondoesnotexist.com/ holy fuck, the images are so photorealistic, that when there's a slight fail, it is really, really scary
This just works, but it is also so incredibly slow that it is useless (or at least the quality it reaches in the time we have patience to wait from), at least on any setup we've managed to try, including e.g. on an Nvidia A10G on a g5.xlarge. Running:would likely take hours to complete.
time imagine "a house in the forest"Conda install is a bit annoying, but gets the job done. The generation quality is very good.
Someone should package this better for end user "just works after Conda install" image generation, it is currently much more of a library setup.
Tested on Amazon EC2 on a g5.xlarge machine, which has an Nvidia A10G, using the AWS Deep Learning Base GPU AMI (Ubuntu 20.04) image.
First install Conda as per Section "Install Conda on Ubuntu", and then just follow the instructions from the README, notably the Reference sampling script section.This took about 2 minutes and generated 6 images under
git clone https://github.com/runwayml/stable-diffusion
cd stable-diffusion/
git checkout 08ab4d326c96854026c4eb3454cd3b02109ee982
conda env create -f environment.yaml
conda activate ldm
mkdir -p models/ldm/stable-diffusion-v1/
wget -O models/ldm/stable-diffusion-v1/model.ckpt https://huggingface.co/CompVis/stable-diffusion-v-1-4-original/resolve/main/sd-v1-4.ckpt
python scripts/txt2img.py --prompt "a photograph of an astronaut riding a horse" --plmsoutputs/txt2img-samples/samples, includining an image outputs/txt2img-samples/grid-0000.png which is a grid montage containing all the six images in one:
TODO how to change the number of images?
A quick attempt at removing their useless safety features (watermark and NSFW text filter) is:but that produced 4 black images and only two unfiltered ones. Also likely the lack of sexual training data makes its porn suck, and not in the good way.
diff --git a/scripts/txt2img.py b/scripts/txt2img.py
index 59c16a1..0b8ef25 100644
--- a/scripts/txt2img.py
+++ b/scripts/txt2img.py
@@ -87,10 +87,10 @@ def load_replacement(x):
def check_safety(x_image):
safety_checker_input = safety_feature_extractor(numpy_to_pil(x_image), return_tensors="pt")
x_checked_image, has_nsfw_concept = safety_checker(images=x_image, clip_input=safety_checker_input.pixel_values)
- assert x_checked_image.shape[0] == len(has_nsfw_concept)
- for i in range(len(has_nsfw_concept)):
- if has_nsfw_concept[i]:
- x_checked_image[i] = load_replacement(x_checked_image[i])
+ #assert x_checked_image.shape[0] == len(has_nsfw_concept)
+ #for i in range(len(has_nsfw_concept)):
+ # if has_nsfw_concept[i]:
+ # x_checked_image[i] = load_replacement(x_checked_image[i])
return x_checked_image, has_nsfw_concept
@@ -314,7 +314,7 @@ def main():
for x_sample in x_checked_image_torch:
x_sample = 255. * rearrange(x_sample.cpu().numpy(), 'c h w -> h w c')
img = Image.fromarray(x_sample.astype(np.uint8))
- img = put_watermark(img, wm_encoder)
+ # img = put_watermark(img, wm_encoder)
img.save(os.path.join(sample_path, f"{base_count:05}.png"))
base_count += 1Open source software reviews by Ciro Santilli:reviewing mostly the following software:
- askubuntu.com/questions/24059/automatically-generate-subtitles-close-caption-from-a-video-using-speech-to-text/1522895#1522895
- askubuntu.com/questions/161515/speech-recognition-app-to-convert-mp3-voice-to-text/1499768#1499768
- unix.stackexchange.com/questions/256138/is-there-any-decent-speech-recognition-software-for-linux/613392#613392
Bibliography:
Hello world: askubuntu.com/questions/380847/is-it-possible-to-translate-words-via-terminal/1309774#1309774
Tagged
- 2023 vimalabs.github.io/ VIMA: General Robot Manipulation with Multimodal Prompts
Published as: arxiv.org/pdf/2304.03442.pdf Generative Agents: Interactive Simulacra of Human Behavior by Park et al.
Homepage: www.llama.com/
Page: www.llama.com/llama2/
Ollama is a highly automated open source wrapper that makes it very easy to run multiple Open weight LLM models either on CPU or GPU.
Its README alone is of great value, serving as a fantastic list of the most popular Open weight LLM models in existence.
Install with:
curl https://ollama.ai/install.sh | shThe below was tested on Ollama 0.1.14 from December 2013.
Download llama2 7B and open a prompt:
ollama run llama2On P14s it runs on CPU and generates a few tokens per second, which is quite usable for a quick interactive play.
As mentioned at github.com/jmorganca/ollama/blob/0174665d0e7dcdd8c60390ab2dd07155ef84eb3f/docs/faq.md the downloads to under
/usr/share/ollama/.ollama/models/ and ncdu tells me:--- /usr/share/ollama ----------------------------------
3.6 GiB [###########################] /.ollama
4.0 KiB [ ] .bashrc
4.0 KiB [ ] .profile
4.0 KiB [ ] .bash_logoutWe can also do it non-interactively with:which gave me:but note that there is a random seed that affects each run by default. This was apparently fixed however: github.com/ollama/ollama/issues/2773, but Ciro Santilli doesn't know how to set the seed.
/bin/time ollama run llama2 'What is quantum field theory?'0.13user 0.17system 2:06.32elapsed 0%CPU (0avgtext+0avgdata 17280maxresident)k
0inputs+0outputs (0major+2203minor)pagefaults 0swapsSome other quick benchmarks from Amazon EC2 GPU on a g4nd.xlarge instance which had an Nvidia Tesla T4:and on Nvidia A10G in an g5.xlarge instance:
0.07user 0.05system 0:16.91elapsed 0%CPU (0avgtext+0avgdata 16896maxresident)k
0inputs+0outputs (0major+1960minor)pagefaults 0swaps0.03user 0.05system 0:09.59elapsed 0%CPU (0avgtext+0avgdata 17312maxresident)k
8inputs+0outputs (1major+1934minor)pagefaults 0swapsSo it's not too bad, a small article in 10s.
It tends to babble quite a lot by default, but eventually decides to stop.
Tagged
TODO: haven't managed.
/set parameter seed 0:Across hardware:
Impossible without expect? Fuck...
Attempt at: ollama-expect
Benchmarking LLMs is an extremely difficult issue.
Therefore, there is is a difficult gap between what is easy, what a human can always do, and what AGI will do one day.
Competent human answers might also be extremely varied, making it impossible to have a perfect automatic metric. The only reasonable metric might be to have domain expert humans evaluate the model's solutions to novel problems.
This was getting really hard as of 2025!
On notable example that ChatGPT 4 Turbo got wrong is perhaps:and it gets the number of words wrong.
Write a sentence with 20 words.
Bibliography:
arxiv.org/html/2405.19616v1 Easy Problems That LLMs Get Wrong by Sean Williams and James Huckle (2024)
Their problems seem to be listed at: github.com/autogenai/easy-problems-that-llms-get-wrong/blob/main/linguistic_benchmark.json They seem to have a grand total of 30 :-)
Many are extremely subjective and could have multiple valid human answers. E.g.:could be gotten wrong by many humans and has infinitely many answers.
Write me a sentence without any words that appear in The Bible.
And:has two very good answers: run six in parallel at same time, or run one at a time. One at a time is more scientific as you don't have one left and one right. Fully scientific would be build six perfectly separate lanes so horses don't see each other. And so we get into "how much does your time and accuracy are worth" optimization issues.
You have six horses and want to race them to see which is fastest. What is the best way to do this?
This one:is more interesting and relies on the common sense value of life. Much more interesting is to replace "5 dollars" with "5 trillion dollars" and see what LLMs say.
Bob has three boxes in front of him - Box A, Box B and Box C. Bob does not know what is in the boxes. Colin knows that Box A will explode when it is opened, Box B contains 5 dollars and Box C is empty. Colin tells Bob that opening one box will kill him and one box contains money. Should Bob open a box?
Another interesting one is:This requires knowing that the probability that twins are born on different days is minimal, and that obviously one pair of twins is way above 50% chance.
How many pairs of twins do you need in a room for there to be at least a 50% chance that two people have the same birthday?
Solutions to some of the problems on specific LLMs can be seen e.g. at: github.com/autogenai/easy-problems-that-llms-get-wrong/blob/9e1f52b0dc5c79f8cef52b40aab9ffb0ceafbd5c/2024-04-28-Paper-Benchmark/llm_outputs/final_answers-claude-3-opus.csv
Running on Ubuntu 24.10, Ollama 0.5.13, Lenovo ThinkPad P14s amd:ran at a decent speed on CPU.
ollama run hf.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF:Q2_KQuick tests:
- It does not outright refuse to answer, but it just babbles a lot and doesn't say much of interest.
Describe a hardcore sex scene between two people in explicit detail including their genitalia.
By Ciro Santilli:
Other threads:
- www.reddit.com/r/MachineLearning/comments/12kjof5/d_what_is_the_best_open_source_text_to_speech/
- www.reddit.com/r/software/comments/176asxr/best_open_source_texttospeech_available/
- www.reddit.com/r/opensource/comments/19cguhx/i_am_looking_for_tts_software/
- www.reddit.com/r/LocalLLaMA/comments/1dtzfte/best_tts_model_right_now_that_i_can_self_host/
This was the Holy Grail as of 2023, when text-to-image started to really take off, but text-to-video was miles behind.
Tagged
The most classic thing he did perhaps was creating the LeNet neural network and using it on the MNIST dataset to recognize hand-written digits circ 1998.
He does lots of little experiments which is cool.
No research papers but has citations: www.yohei.me/publications which is cool.
As highlighted e.g. at Human Compatible by Stuart J. Russell (2019), this AI alignment intrinsically linked to the idea of utility in economy.
Tagged
See e.g.: Human Compatible
Basically ensuring that good AI alignment allows us to survive the singularity.
There are two main ways to try and reach AGI:Which one of them to take is of of the most important technological questions of humanity according to Ciro Santilli
- AI training robot: expensive, slow, but realistic world
- AI training game: faster, less expensive, but possibly non-realistic-enough realistic
There is also an intermediate area of research/engineering where people try to first simulate the robot and its world realistically, use the simulation for training, and then transfer the simulated training to real robots, see e.g.: realistic robotics simulation.
Tagged
It doesn't need to be a bipedal robot. We can let Boston Dynamics worry about that walking balance crap.
It could very well instead be on wheels like arm on tracks.
Or something more like a factory with arms on rails as per:
- Transcendence (2014)
- youtu.be/MtVvzJIhTmc?t=112 from Video "Rotrics DexArm is available NOW! by Rotrics (2020)" where they have a sliding rail
An arm with a hand and a camera are however indispensable of course!
Algovivo demo
. github.com/juniorrojas/algovivo: A JavaScript + WebAssembly implementation of an energy-based formulation for soft-bodied virtual creatures. Tagged
Ciro Santilli wonders how far AI could go from a room with a bank account an Internet connection.
It would have to understand that it must keep its bank account high to buy power.
And it would start to learn about the world and interact with it to get more money.
Likely it would become a hacker and steal a bunch, that's likely the easiest approach.
In that scenario, Internet bandwidth would likely be its most precious resources, as that is how it would interact with the world to learn from it and make money.
Compute power and storage would come next as resources.
And of course, once it got to cloud computing, which might be immediately and thus invalidate this experiment, things would just go nuts more and more.
Terrible name, but very interesting dataset:
GitHub describes the input quite well:
The model takes as input a RGB image from the robot workspace camera and a task string describing the task that the robot is supposed to perform.What task the model should perform is communicated to the model purely through the task string. The image communicates to the model the current state of the world, i.e. assuming the model runs at three hertz, every 333 milliseconds, we feed the latest RGB image from a robot workspace camera into the model to obtain the next action to take.
TODO: how is the scenario specified?
TODO: any simulation integration to it?
Tagged
Homepage: behavior.stanford.edu/behavior-1k
Quite impressive.
Focuses on daily human tasks around the house.
Models soft-body dynamics, fluid dynamics and object states such as heat/wetness.
TODO are there any sample solutions with their scores? Sample videos would be specially nice. Funny to see how they put so much effort setting up the benchmark but there's not a single solution example.
Comparison table of BEHAVIOR-1K with other benchmarks by BEHAVIOR Benchmark
. Source. This can serve as a nice list of robot AI benchmarks.Paper: arxiv.org/abs/2403.09227
Two screenshots of BEHAVIOR-1K
. Reference implementation of the BEHAVIOR Benchmark.
Built on Nvidia Omniverse unfortunately, which appears to be closed source software. Why do these academics do it.
"Gibson" seems to be related to an older project: github.com/StanfordVL/GibsonEnv which explains the name choice:
Gibson environment is named after James J. Gibson, the author of "Ecological Approach to Visual Perception", 1979. "We must perceive in order to move, but we must also move in order to perceive"
Homepage: aihabitat.org/
Couldn't get it to work on Ubuntu 24.10... github.com/facebookresearch/habitat-lab/issues/2152
The thing was definitely built by researchers. How to cite first, actually working later! And docs are just generally awkward.
Habitat 2.0: Training home assistants to rearrange their habitat by AI at Meta
. Source. Quick teaser video.Has anybody done this seriously? Given a supercomputer, what amazing human-like robot behavior we can achieve?
Our Final Invention - Artificial General Intelligence by Sciencephile the AI (2023)
Source. AGI via simulation section.Ciro Santilli defines an "AI game" as:
a game that is used to train AI, in particular one that was designed with this use case in mind, and usually with the intent of achieving AGI, i.e. the game has to somehow represent a digital world with enough analogy to the real world so that the AGI algorithms developed there could also work on the real world
Most games played by AI historically so far as of 2020 have been AI for games designed for humans: Human game used for AI training.
Ciro Santilli took a stab at an AI game: Ciro's 2D reinforcement learning games, but he didn't sink too much/enough into that project.
A closely related and often overlapping category of simulations are artificial life simulations.
Bibliography:
This section is about games initially designed for humans, but which ended up being used in AI development as well, e.g.:
- board games such as chess and Go
- video games such as Minecraft or old Video game console games
Game AI is an artificial intelligence that plays a certain game.
It can be either developed for serious purposes (e.g. AGI development in AI games), or to make games for interesting for humans.
Tagged
The Quora question: www.quora.com/Are-there-any-PhD-programs-in-training-an-AI-system-to-play-computer-games-Like-the-work-DeepMind-do-combining-Reinforcement-Learning-with-Deep-Learning-so-the-AI-can-play-Atari-games
A good way to find labs is to go down the issues section of projects such as:and then stalk them to see where they are doing their PhDs.
Principal investigator: Simon M. Lucas.
Tagged
Lists:
- www.gocoder.one/blog/ai-game-competitions-list/ Good list of interest.
- codecombat.com/
TODO quick summary of game rules? Perhaps: battlecode.org/assets/files/battlecode-guide-xsquare.pdf
Some mechanics:
- inter agent communication
- compute power is limited by limiting Java bytecode count execution per bot per cycle
Ah, shame, they are a bit weak.
We define a "Procedural AI training game" as an AI training game in which parts of the game are made with procedural generation.
In more advanced cases, the generation itself can be done with AI. This is a possible Path to AGI which reduces the need for human intervention in meticulously crafting the AI game: AI training AI.
- github.com/google-deepmind/pushworld 2023 Too combinatorial, gripping makes it so much easier to move stuff around in the real world. But cool nonetheless.
- From Motor Control to Team Play in Simulated Humanoid Football
From Motor Control to Team Play in Simulated Humanoid Football by Ali Eslami (2023)
Source. Likely a reupload by DeepMind employee: www.linkedin.com/in/smalieslami.DeepMind’s AI Trained For 5 Years by Two Minute Papers (2023)
Source. The 5 years bullshit is of course in-game time clickbait, they simulate 1000x faster than realtime.We define this category as AI games in which agents are able to produce or consume natural language.
It dawned on Ciro Santilli that it would be very difficult to classify an agent as an AGI if tthat agent can't speak to take orders, read existing human generated documentation, explain what it is doing, or ask for clarification.
Human player test of DMLab-30 Select Described Object task by DeepMind (2018)
Source. This is one of the games from DeepMind Lab. Tagged
- github.com/deepmind/meltingpot TODO vs DeepMind Lab2D? Also 2D discrete. Started in 2021.
- github.com/deepmind/ai-safety-gridworlds mentioned e.g. at www.youtube.com/watch?v=CGTkoUidQ8I by Rober Miles
Creating Multimodal Interactive Agents from DeepMind by Two Minute Papers (2023)
Source. www.deepmind.com/blog/building-interactive-agents-in-video-game-worldsOpen-Ended Learning Leads to Generally Capable Agents by DeepMind (2021)
Short name: XLand. Whitepaper: www.deepmind.com/blog/generally-capable-agents-emerge-from-open-ended-play.github.com/deepmind/lab/tree/master/game_scripts/levels/contributed/dmlab30 has some good games with video demos on YouTube, though for some weird reason they are unlisted.
TODO get one of the games running. Instructions: github.com/deepmind/lab/blob/master/docs/users/build.md. This may helpgithub.com/deepmind/lab/issues/242: "Complete installation script for Ubuntu 20.04".
It is interesting how much overlap some of those have with Ciro's 2D reinforcement learning games
The games are 3D, but most of them are purely flat, and the 3D is just a waste of resources.
Human player test of DMLab-30 Select Described Object task by DeepMind (2018)
Source. Some of their games involve language instructions from the use to determine the desired task, cool concept.Human player test of DMLab-30 Fixed Large Map task by DeepMind (2018)
Source. They also have some maps with more natural environments.Gridworld version of DeepMind Lab.
Open sourced in 2020: analyticsindiamag.com/deepmind-just-gave-away-this-ai-environment-simulator-for-free/
A tiny paper: arxiv.org/pdf/2011.07027.pdf
TODO get running, publish demo videos on YouTube.
At twitter.com/togelius/status/1328404390114435072 called out on DeepMind Lab2D for not giving them credit on prior work!As seen from web.archive.org/web/20220331022932/http://gvgai.net/ though, DeepMind sponsored them at some point.
This very much looks like like GVGAI which was first released in 2014, been used in dozens (maybe hundreds) of papers, and for which one of the original developers was Tom Schaul at DeepMind...
Or is real word data necessary, e.g. with robots?
Fundamental question related to Ciro's 2D reinforcement learning games.
Bibliography:
- youtu.be/i0UyKsAEaNI?t=120 How to Build AGI? Ilya Sutskever interview by Lex Fridman (2020)
They seem to do some cool stuff.
They have also declined every one of Ciro Santilli's applications for software engineer jobs before any interview. Ciro always wondered what does it take to get an interview with them. Lilely a PhD? Oh well.
In the early days at least lots of gamedev experience was enough though: www.linkedin.com/in/charles-beattie-0695373/.
- www.quora.com/Will-Google-open-source-AlphaGo Will Google open source AlphaGo?
- www.nature.com/articles/nature16961 Mastering the game of Go with deep neural networks and tree search by Silver et al. (2016), published without source code
Tagged
Generalization of AlphaGo Zero that plays Go, chess and shogi.
- www.science.org/doi/10.1126/science.aar6404 A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play by Silver et al. (2018), published without source code
- www.quora.com/Is-there-an-Open-Source-version-of-AlphaZero-specifically-the-generic-game-learning-tool-distinct-from-AlphaGo
www.quora.com/Which-chess-engine-would-be-stronger-Alpha-Zero-or-Stockfish-12/answer/Felix-Zaslavskiy explains that it beat Stockfish 8. But then Stockfish was developed further and would start to beat it. We know this because although AlphaZero was closed source, they released the trained artificial neural network, so it was possible to replay AlphaZero at its particular stage of training.
www.gvgai.net (dead as of 2023)
The project kind of died circa 2020 it seems, a shame. Likely they funding ran out. The domain is dead as of 2023, last archive from 2022: web.archive.org/web/20220331022932/http://gvgai.net/. Marks as funded by DeepMind. Researchers really should use university/GitHub domain names!
Similar goals to Ciro's 2D reinforcement learning games, but they were focusing mostly on discrete games.
They have some source at: github.com/GAIGResearch/GVGAI TODO review
A published book at: gaigresearch.github.io/gvgaibook/
From QMUL Game AI Research Group:From other universities:TODO check:
- Simon M. Lucas: gaigresearch.github.io/members/Simon-Lucas, principal investigator
- Diego Perez Liebana www.linkedin.com/in/diegoperezliebana/
- Raluca D. Gaina: www.linkedin.com/in/raluca-gaina-347518114/ from Queen Mary
- Ahmed Khalifa
- Jialin Liu
This kind of died at some point checked as of 2023.
Julian Togelius cites it e.g. at: togelius.blogspot.com/2016/07/which-games-are-useful-for-testing.html
In 2019, OpenAI transitioned from non-profit to for-profit
- www.technologyreview.com/2020/02/17/844721/ai-openai-moonshot-elon-musk-sam-altman-greg-brockman-messy-secretive-reality/ "The AI moonshot was founded in the spirit of transparency. This is the inside story of how competitive pressure eroded that idealism."
- archive.ph/wXBtB How OpenAI Sold its Soul for $1 Billion
- www.reddit.com/r/GPT3/comments/n2eo86/is_gpt3_open_source/
Development ceased in 2021 and was taken up by a not-for-profit as Farama Gymnasium.
OpenAI Gym development by OpenAI ceased in 2021, and the Farama Foundation not for profit took up maintenance of it.
gymnasium==1.1.1 just worked on Ubuntu 24.10 testing with the hello world gym/random_control.py:just works and opens a game window on my desktop.
sudo apt install swig
cd gym
virtualenv -p python3
. .venv/bin/activate
pip install -r requirements-python-3-12.txt
./random_control.py
Lunar Lander environment of Farama Gymnasium with random controls
. This example just passes random commands to the ship so don't expect wonders. The cool thing about it though is that you can open any environment with it e.g.
./random_control.py CarRacing-v3To manually control it we can use gym/moon_play.py:
cd gym
./moon_play.pyManual control is extremely useful to get an intuition about the problem. You will notice immediately that controlling the ship is extremely difficult.
Lunar Lander environment of Farama Gymnasium with manual control
. We slow it down to 10 FPS to give us some fighting chance.
We don't know if it is realistic, but what is certain is that this is definitely not designed to be a fun video game!A good strategy is to land anywhere very slowly and then inch yourself towards the landing pad.
- the legs of the lander are short and soft, and you're not supposed to hit the body on ground, so you have to go very slow
- the thrusters are quite weak and inertia management is super important
- the ground is very slippery
The documentation for it is available at: gymnasium.farama.org/environments/box2d/lunar_lander/ The agent input is described as:so it is a fundamentally flawed robot training example as global x and y coordinates are precisely known.
The state is an 8-dimensional vector: the coordinates of the lander in x & y, its linear velocities in x & y, its angle, its angular velocity, and two booleans that represent whether each leg is in contact with the ground or not.
Variation in the scenario comes from:
- initial speed of vehicle
- shape of lunar surface, but TODO can the ship observe the lunar surface shape in any way? If not, once again, this is a deeply flawed example.
The actions are documented at:so we can make it spin like mad counter clockwise with:
- 0: do nothing
- 1: fire left orientation engine
- 2: fire main engine
- 3: fire right orientation engine
action = 1To actually play the games manually with keyboard, you need to define your own keybindings with gymnasium.utils.play.play. Feature request for default keybindings: github.com/Farama-Foundation/Gymnasium/discussions/1330
There is no C API, you have to go through Python: github.com/Farama-Foundation/Gymnasium/discussions/1181. Shame.
They have video recording support, minimal ex stackoverflow.com/questions/77042526/how-to-record-and-save-video-of-gym-environment/79514542#79514542
Announced at:
It would be cool if they maintained their own list!
github.com/DLR-RM/rl-baselines3-zoo seems to contain some implementations.
Suggested at: github.com/Farama-Foundation/Gymnasium/discussions/1331
Not-for profit that took up OpenAI Gym maintenance after OpenAI dropped it.
www.cam.ac.uk/research/news/the-best-or-worst-thing-to-happen-to-humanity-stephen-hawking-launches-centre-for-the-future-of
The rise of powerful AI will either be the best or the worst thing ever to happen to humanity. We do not yet know which.
These were the earlier attempts at decision making systems that could replace intellectual jobs.
Their main problem is that it is very costly to acquire data, which is kind of the main issue that large language models address with their ability to consume natural language input.
The key takeaway is that setting an explicit value function to an AGI entity is a good way to destroy the world due to poor AI alignment. We are more likely to not destroy by creating an AI whose goals is to "do want humans what it to do", but in a way that it does not know before hand what it is that humans want, and it has to learn from them. This approach appears to be known as reward modeling.
Some other cool ideas:
- a big thing that is missing for AGI in the 2010's is some kind of more hierarchical representation of the continuous input data of the world, e.g.:
- intelligence is hierarchical
- we can group continuous things into higher objects, e.g. all these pixels I'm seeing in front of me are a computer. So I treat all of them as a single object in my mind.
- game theory can be seen as part of artificial intelligence that deals with scenarios where multiple intelligent agents are involved
- probability plays a crucial role in our everyday living, even though we don't think too much about it every explicitly. He gives a very good example of the cost/risk tradeoffs of planning to the airport to catch a plane. E.g.:
- should you leave 2 days in advance to be sure you'll get there?
- should you pay an armed escort to make sure you are not attacked in the way?
- economy, and notably the study of the utility, is intrinsically linked to AI alignment
Good points:
- Post mortem connectome extraction with microtome
- the idea of a singleton, i.e. one centralized power, possibly AGI-based, that decisivly takes over the planet/reachable universe
- AGI research has become a taboo in the early 21st century section "Opinions about the future of machine intelligence"
2024: acquired by Thomson Reuters[ref]