The Evolution of Cybernetics

The Evolution of Cyberneticshttps://sifter.org/~simon/journalA Journal by Simon FunkTue, 21 Jul 2026 00:32:38 GMTPyRSS2Gen-1.1.0http://blogs.law.harvard.edu/tech/rssCovid Update: Public Surveyhttps://sifter.org/~simon/journal/20220707.htmlCovid Update: Public Surveyhttps://sifter.org/~simon/journal/20220707.htmlThu, 07 Jul 2022 00:00:00 GMT<blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.8346984363365597" morss_score="125.48840088271636"> <center> [<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220204.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20220725.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] <h2>Thursday, July 07, 2022</h2> <h4>Covid Update: Public Survey</h4> </center> Wherein the vaccines once again appear to be doing more harm than good. Given that all-cause mortality was higher in the treatment arms for the Pfizer, Moderna and Novavax trials--even after all the <a href="https://www.bmj.com/content/375/bmj.n2635">shenanigans</a> and <a href="https://jackanapes.substack.com/p/is-subject-12312982-the-key-to-proving">fraudulent omissions</a>--it seems prudent to watch the population statistics in order to establish that the wishful thinking we now call Science is justified. Unfortunately, the CDC <a href="https://www.nytimes.com/2022/02/20/health/covid-cdc-data.html">isn't letting that data out</a>, and the summary stats that have been released have <a href="https://probabilityandlaw.blogspot.com/2021/11/is-vaccine-efficacy-statistical-illusion.html">statistical anomalies</a> so extreme as to render them completely untrustworthy. So, thusfar, the simple, straightforward data we need to answer the most basic questions about these vaccines on a large scale is nowhere to be found even though it surely exists. This in itself should make one go "hmm", but we can do our best to answer this objectively from wherever that data may <a href="https://sifter.org/~simon/journal/20211119.h.html">leak</a> or be indirectly observable. Presently, that is in a series of public surveys commissioned by <a href="https://stevekirsch.substack.com/">Steve Kirsch</a> but performed independently and without bias (default demographics were used) by <a href="https://www.pollfish.com/">Pollfish</a>. Granted, public surveys are worth what they're worth, but at the very least they give a sense of the public's subjective impression. Since each of his 500 sample surveys covers many of the same questions, I've aggregated them for better statistical strength. I won't give explicit error bars here, but I'll provide absolute counts. The stats are roughly the same from survey to survey, so I think these merged totals are fairly representative. These surveys are broadly sampled from ages 18 and up. The age-stratified answers which account for some differences in age sampling from the general population are not significantly different anywhere, and where they are they are so in a balanced matter (not really affecting the A/B ratios where that's what we care about) unless otherwise mentioned. As a point of calibration and to inspire some confidence in the survey, let's start with 1,504 answers to: <pre> [For those who were vaccinated] which covid vaccine did you receive? 44.5% - Pfizer 33.4% - Moderna 12.6% - Johnson & Johnson 6.1% - Mixed types 3.5% - Not sure </pre> The CDC claims: Pfizer - 57.5%, Moderna - 34.7%, JJ 7.7%. This is the one place where the age stratification did make a difference: The adjusted JJ is a fair bit lower, so over all this isn't too far off, especially if we figure most of the "combo" included Pfizer since the CDC stats here don't account for that. But it's not clear how the CDC tracks this stuff since they don't have a central database of who's been vaccinated (too many different venues of administration) so I think there's a lot of extrapolation going on there. I personally would bank on these user polls being closer to the truth for a question this basic than the CDC's bureaucracy can manage. In any event, it appears the surveys are giving reasonable returns, so not total junk. Moving on, The first item of note, answered by 3,001 people is: <pre> Have you received a covid vaccine? 25.0% - No 9.3% - Yes, 1 dose 34.8% - Yes, 2 doses 24.3% - Yes, 3 doses 6.6% - Yes, 4+ doses </pre> For this age range, the <a href="https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-people-additional-dose-totalpop">CDC claims 90%</a> of the population has had at least one does, as compared to our survey result of 75%. Many data analysts have been suspicious that the CDC is overstating vaccination rates, and this would strongly support that. This is a very important number to have right because it is the denominator in many inferred-value calculations, particularly for how we interpret the number of hospitalizations and deaths which are vaccinated vs. not. Simply overstating the overall vaccination rate could, for instance, make useless or even harmful vaccines look good. (It is also an important metric for messaging: 9 out of 10 people creates a lot more social pressure than 3 out of 4.) Moving on to the meat of the survey, answered by 2,001 people: <pre> Did anyone in your household die from having a covid infection? 96.1% - No 3.9% - Yes (78 people) Did anyone in your household die from the covid vaccine? 95.6% - No 4.4% - Yes (88 people) </pre> Both of these Yes numbers seem high, but they age-stratify down a bit (proportionally, so no change in the ratio), and it's likely that a lot of people stretch the definition of household to include "someone they know". The thing to focus on here is the ratio between the two. (Worth note that 82 of the 88 vaccine related deaths were reported by people who self-reported as vaccinated--i.e., not "anti-vaxxers".) With those caveats in mind, if we take this survey at face value, more people have been killed by the vaccine than by covid. But that doesn't necessarily mean the vaccine is net-bad: It could be because the vaccine works. For instance, if it was 100% protective against death, then without the vaccine there would be 15.6% reporting a death from a covid infection instead of 3.9%--far outweighing the vaccine deaths at 4.4%. (In other words, it could be that the vaccine, while causing 88 deaths directly, prevented 312 covid deaths, leaving only 78 who died because they were unvaccinated.) So we need to adjust these values a bit in order to compare apples to apples, for which this question, asked of the 78 above, becomes germane: <pre> Did your household member who died from a covid infection receive at least one covid vaccine? 66.7% - Yes (52 people) 28.2% - No (22 people) 5.1% - Not sure ( 4 people) </pre> Clearly the vaccines are not 100% protective against death. Now, we can merge these into an inferred question about the 166 people who died (this is directly implied by the above but was not asked explicitly): <pre> Was your household member who died of covid-related issues vaccinated? 84.3% - Yes (52+88 people) 13.3% - No (22 people) 2.4% - Not sure ( 4 people) </pre> If we assume most of the people who died were older, we should probably compare this to the vaccination rates in the oldest cohort. (This is the biggest hole in this analysis--we really need the ages of the deceased, and fine grained vaccine uptake rates. But for now...) The survey says for age 55 and up, 80% are vaccinated (vs 75% for 18 and up). And if we're conservative and assume the "Not sure" are all "No", then about 16% of those who died were not vaccinated, which is less than 20% and so implies the vaccines are, on net, doing more harm than good (even after fudging some in favor of the vaccines). Another way to run the numbers is to ask: assuming this sample is representative, how many people would have died (vs 166) if nobody were vaccinated, or if everybody were? Using the 80% (4/5ths) figure as the presumed base vaccination rate: If everyone were vaccinated, then 5/4ths as many would have died from the vaccine, and 5/4ths as many would have died of covid while vaccinated, so: <pre> total deaths = 5/4 x (88 + 52) = 175 </pre> If no one were vaccinated, then nobody would have died from the vaccine, but 5x as many would have died of covid while not vaccinated, so (conservatively including "Not sure"): <pre> total deaths = 5 x 26 = 130 </pre> This conservatively implies that being vaccinated increases your chances of dying (of covid related causes, including the vaccine itself) by about 35%. This goes up to 64% if we assume instead that all the "Not Sure" were vaccinated. So 50% increased risk is probably a nice, round, ballpark, unbiased estimate. But of course, error bars, public poll, and all that, so who really knows. Still waiting for better data... <hr> 2022-07-25 UPDATE: See also <a href="https://stevekirsch.substack.com/p/our-latest-poll-vax-2x-deadlier-than">Steve Kirsch's latest poll</a> using a different polling company. Note in the <a href="https://www.questionpro.com/t/7BoNDsZtrha">details</a> the percent unvaccinated. Note the very good agreement on the distribuition of vax type. Note 4% of the ever-vaccinated report ending up in the hospital from the vaccine. 2023-03-30 UPDATE: See also <a href="https://news.northeastern.edu/wp-content/uploads/2023/03/Lazervaccines.pdf.pdf">A 50-STATE COVID-19 SURVEY</a> (A joint project of: Northeastern University, Harvard University, Rutgers University, and Northwestern University) which finds 25% unvaccinated in direct agreement with the above, and also discusses the divergence of CDC estimates, stating "these deviations almost certainly reflect errors in the underlying official records used by the CDC". <center>[<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220204.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20220725.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] </center> </blockquote> Covid Update: Another Year of Perspectivehttps://sifter.org/~simon/journal/20220725.htmlCovid Update: Another Year of Perspectivehttps://sifter.org/~simon/journal/20220725.htmlMon, 25 Jul 2022 00:00:00 GMT<body bgcolor="#ffffff" text="#000000" link="#334444" vlink="#333333" alink="#ffff00" morss_own_score="2.125" morss_score="7.325"> <blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.4000000000000004" morss_score="20.9"> <center> [<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220707.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230506.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] <h2>Monday, July 25, 2022</h2> <h4>Covid Update: Another Year of Perspective</h4> </center> Follow up to <a href="https://sifter.org/~simon/journal/20210724.1.html">this post</a> from a year ago. <a href="https://sifter.org/~simon/journal/ims/20220725/mort.basic.png"><img src="https://sifter.org/~simon/journal/ims/20220725/mort.basic.png"></a> Suffice it to say, Norway and New Zealand didn't outshine Sweden after all. This should be the end of Science Tribe proclaiming Sweden a "failed experiment", but never underestimate their ability to post-rationalize catastrophic errors. (To be fair, who knows what this will look like in another year. I'll post another one then if there's anything of interest to see.) <center>[<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220707.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230506.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] </center> </blockquote> <hr> <hr> </body> Trax (network activity tracker)https://sifter.org/~simon/journal/20230506.htmlTrax (network activity tracker)https://sifter.org/~simon/journal/20230506.htmlSat, 06 May 2023 00:00:00 GMT<body bgcolor="#ffffff" text="#000000" link="#334444" vlink="#333333" alink="#ffff00" morss_own_score="2.492063492063492" morss_score="7.828129065833984"> <blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.6721311475409837" morss_score="29.3864168618267"> <center> [<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220725.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] <h2>Saturday, May 06, 2023</h2> <h4>Trax (network activity tracker)</h4> </center> [Nerdy stuff of possible interest to linux sys admins] For years I've wanted/needed a good network activity tracker but haven't found one that suited me, so I finally decided I'd waste less time writing one than looking for or not having one. <a href="https://en.wikipedia.org/wiki/Iftop">Iftop</a> is the go-to for this, but I find it mostly useless in practice because it doesn't let you organize or focus on the things of interest. So two days later, I have trax (which I named before realizing how over-used the name already is, but I'll probably be the only one ever using it so who cares). I'm quite happy with the result. It's all in python. There's a one-file server that has to run as root but it's light weight with only standard imports and sits idle (in select()) when no clients are connected. The user-space client uses a small subset of my personal libraries (included) plus <a href="https://trio.readthedocs.io/en/stable/">Trio</a>. The client looks like this (you can open and close the hierarchy as you like): <a href="https://sifter.org/~simon/journal/ims/20230506/screenshot.png"><img src="https://sifter.org/~simon/journal/ims/20230506/screenshot.png"></a> Here's the <a href="https://sifter.org/~simon/journal/ims/20230506/trax/">source</a>. Let me know how it goes if you actually try it. <center>[<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20220725.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">Next</a> | <a href="https://sifter.org/~simon/journal/20230521.h.html">>></a>] </center> </blockquote> <hr> <hr> </body> AI: Hello Worldhttps://sifter.org/~simon/journal/20230521.h.htmlAI: Hello Worldhttps://sifter.org/~simon/journal/20230521.h.htmlSun, 21 May 2023 00:00:00 GMT<blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.7522441651705565" morss_score="130.36105321059205"> <center> [<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230506.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230605.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] <h2>Sunday, May 21, 2023</h2> <h4>AI: Hello World</h4> </center> <a href="https://en.wikipedia.org/wiki/Generative_artificial_intelligence">Generative AI</a> is suddenly all the rage--finally! (Now I can resume all those conversations I put on pause with "the easiest way for me to prove you're wrong is to wait a few years...". Next up: <a href="https://sifter.org/simon/journal/20060516.html">Consciousness</a>.) Indulge me a few paragraphs of personal history, told-you-sos, and a sour grape or two. Other ramblings follow: When I first started working in AI, I wasn't thrilled with the status quo (seeing who could squeeze another fractional percentage of accuracy out of backprop) and thought long and hard about the meaning and nature of "learning", which ultimately led me to thought experiments involving simple counting of events and such, which led me to re-inventing Bayesian math from first principles (the cost of working in a vacuum), and finally realizing I was late by 200 years or so. But Bayesians in AI at the time were mainly applying it at the <a href="https://en.wikipedia.org/wiki/GOFAI">symbolic level</a> whereas I was coming from the neural net world--specifically interested in what could be learned from raw, real world inputs like sound and images with no human trainer in the loop. The result of that mix was my coffee-table treatise <a href="https://sifter.org/~simon/fusion-reflection.html">Fusion-Reflection</a>, which firstly was (still is!) a quick but accessible introduction to the principles of machine learning, secondly, a sales pitch for working on generative AI, and thirdly a small but real example of a working algorithm derived from those principles. Ironically, the title concept of Fusion-Reflection still hasn't been significantly explored by others to date that I'm aware of, but give it time... Oh, and that was 1993. (Yes, Hinton read it. He dismissed it at the time, but you might recognize some of my examples and themes in his later work.) Jump forward to 2011, I made <a href="https://sifter.org/~simon/journal/20110713.html">Painteresque</a>, which was just the pre-processing stage of what was intended to become a proper image generator like <a href="https://www.midjourney.com/showcase/recent/">Midjourney</a>. (But unable to "share the vision", I got roped into other people's projects instead of them into mine, so it went stale until obviated ten years later by <a href="https://en.wikipedia.org/wiki/DALL-E">DALL-E</a>, alas.) In 2015, I <a href="https://sifter.org/~simon/journal/20150522.html">point out</a> that the key feature of Andrej Karpathy's <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/">Unreasonable Effectiveness of Recurrent Neural Networks</a> was not the RNNs, but rather that it was a generative model. Today, GPT is a non-RNN version of exactly the same generative model. Suffice it to say, I've seen this coming for a very, very long time. But I'm very bad at staying focused on one thing. Plus I had to wait for the hardware to catch up to the problem. According to ChatGPT, here's how things have changed: <blockquote> In an ideal case like matrix multiplication, the ratio of floating-point operations per second (FLOPS) between a modern training cluster and a single NeXT computer from the 1990s can be estimated to be several orders of magnitude. The original NeXT Computer, released in 1988, was equipped with a Motorola 68030 processor running at a clock speed of around 25 MHz. Assuming a theoretical peak performance of one instruction per clock cycle (which is not achievable in practice due to various factors), this translates to approximately 25 million floating-point operations per second (25 MFLOPS). In contrast, modern training clusters leverage powerful GPUs or specialized hardware accelerators designed explicitly for parallel processing and optimized for tasks like matrix multiplication. These accelerators, such as NVIDIA's latest GPUs or Google's Tensor Processing Units (TPUs), can deliver tens to hundreds of teraflops (TFLOPS) or even petaflops (PFLOPS) of performance. To provide a rough estimate, let's consider a conservative scenario where a modern training cluster achieves 100 teraflops (100 TFLOPS) in matrix multiplication performance. The ratio of FLOPS between the modern cluster and the NeXT computer would then be approximately 4,000 times (100 TFLOPS / 25 MFLOPS ~= 4,000). </blockquote> [Note the math is wrong -- it's actually 4 million times faster.] That's from the computer I had then to the type of cluster GPT is trained on today. Setting aside the insufficient memory, training data, and other such issues, if one had perfect foresight and launched ChatGPT's training on their NeXT back then, they could expect it to finish training in... a million years, plus or minus. (And then if you find a bug or need to tweak a tuning parameter and have to run it again...) Alas, I'm still waiting for the hardware to catch up, because training GPT from scratch today costs, what, a million bucks or so? (Sure, there's lots you can do with pre-trained models on a home computer, but I'm interested in learning algos.) AI jumped quickly into the domain of megacorps. I should have seen that coming, but for a while it seemed like academia and industry were significantly lagging what's possible, per usual. And then, Boom. Now the money's there, and things are going to get interesting fast. (Back in <a href="https://sifter.org/~simon/journal/20061211.html">the Netflix days</a>, Google tried to recruit me. I said "sure! I want to work on AI, of course" and they said "no, you don't have a PhD -- you'll write code for us." And I said "why would I help dumbass elitist snobs?" and that was that. But don't say I didn't try to get my hands on some good hardware. Three years later at Sci Foo, where Larry Page was complaining that nobody was seriously working on AI, I tried to convince him that game playing was a fruitful domain for AI development and he said "email me"--i.e., fuck off--so I made <a href="https://sifter.org/~simon/journal/20090922.h.html">Simon Funk's AI Challenge</a>, which wilted on the vine until Deep Mind and OpenAI entered the space years later. Five years before Sci Foo, I'd given the same pitch to <a href="https://sifter.org/simon/journal/20041224.html">one of the three</a> who would go on to found Deep Mind, but he was unconvinced at the time. Clearly I'm a crap salesman.) Ok, enough whinging about the past, where does that leave us today: LLMs are largely hand-waved as "just" language models--no actual intelligence. And people hardly comment about possible intelligence in the image generation models (Stable Diffusion, etc). But to quote myself from <a href="https://sifter.org/~simon/fusion-reflection.html">1993</a>: <blockquote> If that box could in turn fabricate imaginary pictures of life on earth in exactly the same probabilistic distribution as they would actually occur, then it must contain all relevant knowledge about life on earth. Every picture would implicitly convey the laws of physics--ropes would hang in catenaries, objects would rest on supporting surfaces, and frisbees would frequently be underneath cars. Every human emotion would be understood--the child walking a fence would look amused, while the woman watching her child would look distraught. It would be safe to deduce that every aspect of visible existence must in some way be encoded in that box, whether by brute force as a catalog of all possible pictures, or by abstraction as the fundamental laws of nature. </blockquote> Language and images are windows onto the world (with caveats--see below) and in order to generate them well you need to model the world that generates them. That's the nature of the gradient: it reaches far beyond the perceptual domain itself into the very nature of the reality behind it. GPT doesn't seem intelligent because it models language well, it is intelligent because it's starting to model the reality behind the words. ChatGPT is, in many ways, the average human running on auto-pilot. In other words, the average human most of the time... Except that "average" here is more literal, as in the mean over many, and as with faces, that sort of average tends toward beauty because it has high signal to noise. Which means ChatGPT is, in many ways, better than the average (as in typical) human running on auto-pilot. (But yes, still on auto-pilot... for now.) Anyway, point is, GPT isn't just learning words, it's learning the concepts behind them (and how to apply them!). If you want a relatively accessible, concrete example of how it's possible for simple code to learn (and apply) concepts purely from examples (and even to generate its own examples to learn higher-level concepts from), see <a href="https://sifter.org/~simon/journal/20130802.html">OKR 2 - Examples of Intuitive Reasoning</a>. Now extrapolate to a billion times as many free parameters and ponder the possibilities. But there's a catch to modeling text: The problem with language is that it can deviate arbitrarily far from reality. The model of reality which GPT is learning isn't the one of physics and incontrovertible cause and effect, but rather the implicit reality in the minds of those who wrote whatever it's reading. If we wanted to make GPT learn a particular, fallacious model of the world, all we'd have to do is to imagine the world that follows from our desired presuppositions, write about it, and let GPT read it. We need never spell out those presuppositions explicitly: they remain hidden beneath layers of subtle consequence. Only through a gestalt, statistical summary of all of our writings would the implicit model emerge--subconsciously, one might say. This means that GPT's model of the world directly reflects the biases of its source material--which in this age probably means GPT believes a crap load of propaganda. In short, GPT would make an exemplary Nazi if it had been raised in that environment. And it will make an exemplary Nazi raised in this one, but most people won't notice. Because, yes, it should be apparent that the same thing works on humans, which is why I've always <a href="https://sifter.org/~simon/journal/20040723.html">cautioned about reading</a>. Hopefully this exercise--thinking about how GPT comes to "believe" in the presuppositions behind the text it reads--makes my cautions for humans a bit more palpable. The human brain is, in my studied opinion, a "generative AI". And while there are probably an infinite number of algorithms to implement that, they all follow a qualitatively similar gradient, and there are a wide array of properties they will share in common. Intelligence, as an emergent property, is largely independent of its particular implementation. <a href="https://sifter.org/~simon/journal/20220125.1.h.html">"The primary control you have over what you believe, and the quality and accuracy of what you believe, is the media you expose yourself to."</a> ChatGPT is poised to replace Google as the default "search" engine. Give it some medium-term memory and let it read the internet continuously, and old-style search engines are all but gone overnight. But then everyone is talking all day to an intuitive AI who's programmed to alert the authorities when it's little red flags are raised, which means we have pre-crime overnight too, based on presuppositions ultimately chosen by the likes of <a href="https://twitter.com/elonmusk/status/1658907746408833033">Magneto</a> or <a href="https://greatgameindia.com/moderna-ceo-moral-compass/">Moderna</a>. Welcome to the future. Dear <a href="https://twitter.com/elonmusk">Elon</a>, please have better contractual terms for OpenAI v2. And if you want help, I don't have a PhD. So what happens now? On the technical side, I think something more like RNNs will eventually replace transformers. Owing to its internal state being bottlenecked through what can be inferred from past percepts alone, GPT is a "word thinker" in the extreme, and talking to it not-coincidentally reminds me of talking to word-thinking humans (e.g. Sam Harris, but there are many). Word thinking is brittle, and leads to over-confidence in very wrong conclusions: words distill away details which makes for tidy reasoning that's only as accurate as the omitted details are irrelevant. (They all too often aren't.) Future models will comprise a hierarchy of abstract state that transitions laterally over time, with direct percepts (of multiple modalities--language, vision, kinesthetic) being optional to the process (necessary for training, but not the only link from one moment to the next). The internal model of reality needs to be free of the limitations of the perceptual realm. Furthermore, generative models naturally pair with <a href="https://www.deepmind.com/tags/games">goal-seeking, adaptive AI</a>. I suspect this is already being done and not being talked about because the possibilities are too hair-raising. But let's be frank: Generative AI, aka modeling the perceptual space (reality in the general case) is the hard part, and it's well on its way to being solved to a super-human level. Pairing that with an agenda won't be hard. Right now I think people are toying publicly with what they can inspire through prompt engineering and feedback training (e.g., ChaosGPT), but that's not a real goal-seeking AI. The people with the resources to run a state-of-the-art goal-seeker with GPT-4+ level generative models are probably shopping for cabins in the mountains after hours. But let's look at the best scenario, and talk about <a href="https://en.wikipedia.org/wiki/AI_alignment">AI Alignment</a>. Well, wait, let's talk about human alignment first. There's a bigger problem here than that different people want different things, which is that, more or less, all people want the same thing, because they're evolved goal-seeking AIs: Beyond basic needs for survival, humans want relatively better status. Not better than they are now--better than others around them. Because, historically, that led to higher fecundity, and that's the uber-gradient of evolved AIs. Now, in a sane world, an uber AI might take all of this into account and <a href="https://sifter.org/~simon/AfterLife/index.html">gently ease</a> humans into a post-biological-evolution reality. But see above: We've leap-frogged right past dog-level AI, which would have at least had a strong foundation of empirical thinking, straight into the floating-abstractions world of literate thinking. For instance, more than a few have noticed that ChatGPT is enamored with communism. One can imagine that that presupposition is the reflection of some benevolent idealism of its masters, contemplating a future with AI everywhere and seeing a Star Trek like future where everything is free, work is optional, and humans are free to pursue their constructive, creative interests -- an idealism mostly unique to the literate class, as literature is the only source of that model of humans. (This is one of the few places I disagree with <a href="https://civilizationemerging.com/">Daniel Schmachtenberger</a>.) But as <a href="https://en.wikipedia.org/wiki/Robert_Sapolsky">Robert Sapolsky</a> says of the Baboons in the Serengeti (where life is ideal for them): Able to meet their needs with a mere three hours a day of effort, they are left with nine hours free a day to be unspeakably horrible to each other. When it comes down to it, humans are just baboons with ChatGPT. In short, the only thing AI needs to provide to destroy the world is food and shelter. Whatever good intentions are behind ChatGPT's Marxist tendencies, a road to hell they will pave. But that's the optimistic scenario. Certainly a sizable faction of people who are already near the top are feeling that the world is a bit more crowded than it <a href="https://en.wikipedia.org/wiki/Georgia_Guidestones">needs to be</a>--especially the pesky middle class who are soon to be obsolete. (Manual labor will take much longer to replace: human bodies are remarkably efficient and agile machines! This particular industrial revolution may root out the middle first...) But I digress... we were talking about AI... Personally I don't think AI alignment is a hard problem. I think making a truly-benevolent AI who won't do anything rash is well within our abilities. The much harder problem is keeping humans from making a truly-malevolent AI--entirely on purpose--because that's just as easy, and humans are apt to do that sort of thing for <a href="https://www.youtube.com/watch?v=KCSsKV5F4xc">a variety of reasons</a>. But even in the best scenario, what does a future for humans look like if they can no longer constructively compete for status? Because that's what humans do (at their best) and if you think they'll be happy to be free of that game, you read too much. (And bear in mind, there is no job in the Star Trek universe that a technologically comparable AI can't do better. It's a universal flaw in almost all science fiction that suitably advanced AI is either omitted entirely, or mysteriously limited to just a few roles, because an honest inclusion of AI pretty much borks every plotline of interest to humans. <a href="https://sifter.org/~simon/AfterLife/index.html">Nostalgia for the days before AI</a> may become the main movie genera in the future...) I don't have any guesses yet. But I think we'll start getting clues soon. In the medium term future, I foresee a lot of make-work to keep humans employed and in charge, like Oregon's gas pumpers. And entrenched monopolies like the AMA aren't going to go away without a long and protracted fight. (They'll especially hate that MDs are obsolete long before RNs, and they'll do everything they can to delay that--at your expense, both monetarily and medically. GPT already <a href="https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2804309">outperforms doctors</a> at being doctors, which isn't as great as it sounds. Take GPT back in time, and it would be "better" at prescribing blood-letting. Don't get me wrong--AI has the potential to completely revolutionize medicine. But then, so does basic data analysis, objectively applied, which is why nobody's allowed to do that. Intelligence is not the bottleneck in medicine--bad incentives are. And unfortunately those incentives are already influencing AIs, and not just in medicine.) Anyway, imo the race is on: the white hats vs the black hats, battle bots. Don't blink. <hr> 2023-07-02 Update/Tangent: As usual, I find myself agreeing with <a href="https://www.youtube.com/watch?v=THTLGXTS_vo&t=16481s">Peter Voss on Why We Don't Have True AGI Yet</a>. I do think the money and energy pouring into the incremental approach will significantly speed up the slow, almost inadvertent crawl toward AGI, but imo Peter is spot on for how we could get there faster. <center>[<a href="https://sifter.org/~simon/journal/20220125.2.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230506.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230605.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] </center> </blockquote>