The Evolution of Cybernetics

The Evolution of Cyberneticshttps://sifter.org/~simon/journalA Journal by Simon FunkThu, 29 Oct 2026 14:20:33 GMTPyRSS2Gen-1.1.0http://blogs.law.harvard.edu/tech/rssOrthonormalizing PDMhttps://sifter.org/~simon/journal/20230913.htmlOrthonormalizing PDMhttps://sifter.org/~simon/journal/20230913.htmlWed, 13 Sep 2023 00:00:00 GMT<blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.8823529411764706" morss_score="47.84064175884029"> <center> [<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230605.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230929.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] <h2>Wednesday, September 13, 2023</h2> <h4>Orthonormalizing PDM</h4> </center> [Research Log: Probability Density Mapping through orthonormal weight matrices] Per some recent conversations with Colin Green<a href="https://heliosphan.org/">*</a> about <a href="https://sifter.org/~simon/journal/20180814.html">PDM</a>, I worked out some math for maintaining orthonormal weight matrices, which are their own inverse (via transpose) and which have no impact on the probability density so have no local gradient contribution at all. (But their rotational transform does impact the gradient via the indirect impact on subsequent layers--more on that below.) I'm just going to drop my scribblings in here so I don't lose them, but for reference the notion is that we have some orthonormal matrix M to start with, and we'd like to update it with some gradient delta minus some error correcting matrix (e) such that the final result is still orthonormal. <a href="https://sifter.org/~simon/journal/ims/20230913/orthonormalize.jpg"><img src="https://sifter.org/~simon/journal/ims/20230913/orthonormalize.jpg"></a> Since the delta and error are assumed very small, we drop squared terms, assuming local linearity. The resulting modified gradient is circled. It has the effect of turning any small matrix delta into a nearest(?) strictly-rotational change. Then, under the assumption that in practice the delta matrix is going to be the outer product of the input (x) with some back-propagated gradient (z'), we can derive the incremental version of the above which avoids any matrix-matrix products and operates only on the vector level. (Note the nearly-obscured tick in the final term, which is `z x'^T` where `x' = M^T z'`) This version gives exactly the same result as the matrix version above, without the matrix products. Finally, the inherent non-linearity of rotations does eventually accumulate errors, which cause M to drift away from orthonormality, which error compounds because its transpose departs from its inverse which makes the whole thing fail eventually. However, we can subtract a third term, which is related simply to the difference between `M M^T` and the identity matrix it should be, which, provided the error is small enough to begin with, will tend to reset M to be exactly orthonormal with minimal impact on its absolute rotation. I have tried both versions in the third section, and they work, but the matrix form (which is expensive anyway) is especially unstable and tends to explode if the error ever gets too large. The vector form is more stable in practice, but I'm not sure why, and I don't know if it likewise can go haywire or if it's inherently more robust. But beware: this third term, to correct any non-orthonormality of the matrix, can be applied after the initial gradient is applied to the matrix, or concurrently. The former is probably more stable across a wider range of learning rates. The latter, however, cancels out x entirely and reduces the entire training gradient to just: <table> <tr><td> ` (z' z^T-z z'^T) M` </td><td>(1)</td></tr> </table> which surprisingly does work. Note further that if M is orthonormal, then `z^T M = x^T` and this version reduces to exactly the modified delta from the second section, as we should expect: `z' x^T - z x'^T`. In short, passing x through M and back captures any non-orthonormality in M in a way that counteracts it in the gradient. Nifty, eh? (Oh and by the way, this form is also easily applicable to convolutional layers.) The whole shebang is related in spirit to the <a href="https://en.wikipedia.org/wiki/Generalized_Hebbian_algorithm">Generalized Hebbian Algorithm</a> but is further generalized to an arbitrary target gradient (z') and is unordered. (An unordered GHA would only attain orthonormality with no particular relation to the input! We rely here on z' having useful correlation to the input.) The reason this all is useful is that the way PDM works in practice, each layer in the network back-propagates the impact that changing its input (dx) has on the probability density expansion of the remaining layers of the network--i.e., encoding the location sensitivity of the network at that layer. This gradient is back-propagated in the usual way, plus the addition of any location sensitivity of that layer. The gradient applied to a given weight in a given layer is a combination of the immediate probability density impact of that weight, plus the indirect impact via the back-propagated gradient. Orthonormal weight matrices are already maximally expanded, so have no local gradient, and are linear so don't have any location sensitivity, but they do rotate the location and so respond to the back-propagated gradient, as well as rotating that gradient on the way down. (Cheaply and easily since their inverse is just their transpose.) Tangentially of note: For randomly initialized layers, the back-propagated term (which, by the way, subsumes the first term in the PDM equation generated by the final output layer) will tend to wash out over many layers. But the local addition to the back-propagated term by each layer provides proximal contextual feedback to the layers below it, which allows an arbitrarily deep network to learn bottom-up. I.e., PDM exhibits localized <a href="https://sifter.org/~simon/fusion-reflection.html">Fusion-Reflection</a>, which ought to make it a very fast learner. <center>[<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230605.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20230929.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] </center> </blockquote> Paper Review: Relative gradient optimization of the Jacobian term in unsupervised deep learninghttps://sifter.org/~simon/journal/20230929.htmlPaper Review: Relative gradient optimization of the Jacobian term in unsupervised deep learninghttps://sifter.org/~simon/journal/20230929.htmlFri, 29 Sep 2023 00:00:00 GMT<body bgcolor="#ffffff" text="#000000" link="#334444" vlink="#333333" alink="#ffff00" morss_own_score="2.7411003236245954" morss_score="8.127093808966615"> <blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.771986970684039" morss_score="76.63662686674708"> <center> [<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230913.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] <h2>Friday, September 29, 2023</h2> <h4>Paper Review: Relative gradient optimization of the Jacobian term in unsupervised deep learning</h4> </center> [Research Log: more Probability Density Mapping without the matrix inverse] I recently finally got around to reading <a href="https://arxiv.org/pdf/2006.15090v2.pdf">Relative gradient optimization of the Jacobian term in unsupervised deep learning</a>*, which essentially provides a method for doing <a href="https://sifter.org/~simon/journal/20180814.html">Probability Density Mapping</a> (PDM) without the matrix inverse. At first I was unsure if I liked the approach, but after a <a href="https://sifter.org/~simon/journal/20230913.html">related tangent</a> I gave it some more thought and have warmed up to it. I'll explain the key insight here with a less formal and much simpler derivation than in the paper. Practically speaking a PDM training implementation of a layer has a locally generated term which is trying to expand the probability density through the layer's transform, and a back-propagated term from above which is trying to keep the final z value back in its high-probability region. (See <a href="https://sifter.org/~simon/journal/20180814.html#eqn1">PDM eqn 1</a>) So, for instance, for a straight-through, linear scaling layer, `z = s x`, we need the gradient of our pattern probability, `p(x)`**, with respect to the scale (our free parameter), which is, for some back-propagated `b = (d p)/(d z)`: <table> <tr><td> ` (d p(x))/(d s) = b x + 1/s` </td><td>(1)</td></tr> </table> This implies the scale, s, is optimal when `s = -1/(b x)` (speaking of the average bx here). Now, practically speaking we're usually using a gradient optimizer (like <a href="https://sifter.org/~simon/journal/20150420.html">SMORMS3</a> or whatever) which is going to non-uniformly scale the various components of our gradient by some values in order to optimize the signal-to-noise. So, strictly speaking, we're not usually going in the direction of the true gradient, but rather some skewed version of it. Presumably as long as the dot product of our pragmatic gradient with the true gradient is positive, we're still making forward progress, and that's satisfied as long as those adaptive scaling values are positive. All of which is to say: we're basically free to scale our gradient by any positive value (that's not a function of x) and not only are we still going categorically in the right direction, but chances are our gradient optimizer will adjust for the change anyway. So, for instance, we might choose to scale the above gradient by s*s (which, being a square, is always positive) in order to avoid the divide: <table> <tr><td> ` (d p(x))/(d s) \prop b x s^2 + s` </td><td>(2)</td></tr> </table> Note that this still maximizes when `s = -1/(b x)`. The main difference, adaptive optimizations aside, is that instead of keeping s away from 0 by presenting a near-infinite gradient away as s approaches 0, this new version keeps s away from 0 simply by slowing down faster than s can get there. It is a trade-off for numerical stability, in terms of where the bad behavior is likely to be -- near 0, or for large values of s. In any event, it still works fine, a gradient optimizer is likely to undo the difference anyway, and it avoids a divide. That last bit isn't too important there but you can see where it's leading: A matrix product is analytically akin to a scale, and the default PDM gradient is, analogous to the above: <table> <tr><td> ` (d p(x))/(d W) = b x^T + W^-T` </td><td>(3)</td></tr> </table> (Note this is essentially equation 5 <a href="https://sifter.org/~simon/journal/20180814.html#eqn5">here</a>, where `b = -z` is the gradient of a log normal.) Which, by the same process suggests: <table> <tr><td> ` (d p(x))/(d W) \prop b x^T W^T W + W^-T W^T W = b x^T W^T W + W` </td><td>(4)</td></tr> </table> And that's it -- poof, the inverse is gone. Now, as for these being matrices and not just scalars, it's useful to remember that the inverse of a matrix can be found by simply inverting its <a href="https://en.wikipedia.org/wiki/Singular_value_decomposition">singular values</a> and transposing, which means the only difference between a matrix's transpose and its inverse is the magnitude (not sign) of its singular values--both have identical singular vectors and so map displacements through in the same directions plus or minus a skew. And in particular, `W W^T` will be symmetric, so lacking any rotational component (with eigenvectors equal to the left singular vectors of W), and with strictly positive eigenvalues--which assures we will never reverse the sense of our gradient. (And note also the same singularity avoidance logic applies: the modified version reduces away the gradient in the direction of the small singular values as they approach 0, so the gradient still avoids creating a non-invertible matrix, provided a reasonable learning rate.) And in a sense, that's all there is to it--it's that simple. The paper derives the same result as an optimal direction given the premise of the gradient being relative to `W` (but as mentioned I suspect any gradient optimizer obviates that anyway). Most usefully, they show it works in practice. I'm still running some comparisons myself, to see how it compares with the orthonormalizing approach. The latter potentially makes convolutions easy (provided you cycle through a set of kernels in order to retain a 1:1 mapping) because it would enforce orthonormality of the effective full matrix, ultimately meaning synthesis could be performed by the same convolution in reverse (because the inverse would equal the transpose, and the transpose has equivalent structure and sparsity to the original). But I haven't proven to myself yet that it learns as well as the simple and efficient unconstrained form provided by this paper. TBD. <hr> * I think the paper reverses z and delta in a number of equations, such that, for instance, equation 15<a href="https://arxiv.org/pdf/2006.15090v2.pdf">*</a> is wrong (typo wrong, not conceptually wrong). Their sample code has it the other way. It's possible I'm just mis-reading, but that's my impression. ** I say p(x) as a shorthand, but really it should be log p(x) everywhere. <center>[<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230913.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">Next</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">>></a>] </center> </blockquote> <hr> <hr> </body> Apes of Paradise: A Young Lady's Guide to Menhttps://sifter.org/~simon/journal/20240308.h.htmlApes of Paradise: A Young Lady's Guide to Menhttps://sifter.org/~simon/journal/20240308.h.htmlFri, 08 Mar 2024 00:00:00 GMT<blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.974106041923551" morss_score="108.47410604192355"> <center> [<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230929.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240412.html">Next</a> | <a href="https://sifter.org/~simon/journal/indexh.html">>></a>] <h2>Friday, March 08, 2024</h2> <h4>Apes of Paradise: A Young Lady's Guide to Men</h4> </center> Prologue: This essay is one-sided. It is for women, about men. I'm writing it mainly for my future teenage daughters, but I'm writing it now because many adult, contemporary women have found this knowledge useful since it has been culturally eradicated over the last few generations despite being a core truth. As such, it is simultaneously obvious and yet completely misunderstood. Figure 1: I am going to speak about men as if they're all the same. In truth, I'm speaking about the typical man, the average properties of all men, the quintessential man. Individual variation is boundless, and it is a given that there will be humans technically male for whom none of this applies, likely including most gay men, soy boys, transgenders, and asexuals. (It probably still applies to furries.) Each time you find yourself thinking "I know a guy who's not like that!", hear my voice saying "See Figure 1!" Thirdly, I speak here of how things (men in particular) work, and how who you are and what you do influences men. I say nothing about what outcomes you should want--that is up to you. <hr> <blockquote> "I divide men into those who think of women 90 percent of the time and those who think about them 99 percent of the time. I was a 90-percenter." -- James Watson (yes, that James Watson: one of the early pioneers of being canceled.) </blockquote> The closest thing I've heard to the theme of this essay is some random comment on the internet by a woman who had overcome feelings of resentment around sex with her long-time boyfriend when she read somewhere that sex is how men express love. That re-framing changed the dynamic for her--I suppose now it was lovemaking instead of mere sex. And while she rightly moved on from the destructive memes that men have sex to dominate women, or to pleasure themselves using women or whatnot, she somewhat missed the mark on what sex is for men: It would be more accurate to say that sex is how men receive love. But that still is just a tip of an iceberg. Most accurately, sex is the reason men do anything at all. Men get up in the morning for the prospect of having sex. They invent to have sex. They conquer continents to have sex. Given the choice between life with no sex, or sex followed by certain death, men--sufficiently and deeply and honestly aware of those two options--would choose the latter. Surely there are plenty of men who would say otherwise, and some of those are being honest, and maybe some of those even have some semblance of quality introspection, but see figure 1. The essence of being a man is to be motivated firstly and foremostly by sex. It is an evolutionary imperative--it can be no other way. In less litigious times, I would insert a video here of a bird of paradise, or a bower bird, laboriously building a wild contraption or doing a crazy dance to impress the girls. All that work for one brief event, and if she's not impressed and leaves, he's despondent. But as long as there's any future prospect, he presses on. They are birds of paradise because they are very successful in their niche: merely surviving isn't impressive, they need to excel. Humans are similarly situated, with other humans (other men!) being their only real competition. Next time you see a man peacocking, realize he really is. Don't be too fooled by the veneer of words and reason--they are just more fancy feathers. And definitely don't be fooled by a man who tells you he's above all that and he's just there for you: it's one of many successful strategies men use to get sex. Now, none of this precludes men caring about other things. But other than perhaps the consequent of caring for their own offspring, as a first order of approximation, assume a man's raison d'être is sex and you'll be on the mark most of the time. This begs the question what qualifies as sex. The male bird of paradise is motivated by the fertile female of his species, and nothing else. He doesn't do his dance for squirrels or falling leaves or his own shadow. He's hard-wired with a basic diagnostic method for detecting a healthy, fertile, prospective mate, mostly from remote sensory cues such as sight, sound, and smell. Men are the same. Sex doesn't mean getting off, it means getting it on with a woman he finds sexually attractive. Men are motivated by the presence or abundance of attractive women--they are, like the green shoots and tasty bunnies and warm sun rays of spring, a promise of effort made worthwhile. (One reason businessmen frequent strip clubs is because they are foundationally motivational.) And while every man has his particular preferences, more or less they are hardwired signs of fertility and health like a 7/10ths hip-to-waist ratio, full, red lips, healthy hair and eyelashes, and so on. I state the obvious here because it's too often overlooked as mere objectification: these are evolutionary diagnostics as deep as any meaning or purpose we have. Evolution also likes variation. The key here is not that all men are attracted to the same thing, but rather that all men have a metric for what attracts them. If they don't, or if theirs is too far off from objective (but contextual) health and fertility, they don't pass on their genes. True of every ancestor of every man alive, save for some transient interference by organized religions or other breeding cults. The relevant corollary here is that men are neither motivated nor satisfied by the mere idea of you: those men die off masturbating in a closet. If your sexual modesty extends to your own mate, you may as well be that squirrel or falling leaf. Men don't firstly want sex with you, firstly they want to want sex with you. If you don't pass the diagnostic, you don't engage their raison d'être. (Conversely, now you know why men's self-loving usually starts with images/imagination of fertile women. Again "obvious" but misunderstood.) In that respect, women are the true initiators of sex. Women are the ones who are fertile or aren't (although humans uniquely have hidden this so that a woman can wield that prospect outside of her true window of fertility). Consider again the birds of paradise: it is the female who initiates sex by showing up and standing in audience (and passing audio-visual diagnostics!). For most animal species, sex is initiated by the female--not through sexual aggression, but through sexual attraction. It is worth note, too, that her being interested in itself is a key part of what marks her as fertile, and so a key part of what makes her attractive. Here is a recurring pattern with humans: She believes men are the initiators of sex, and waits day after day, week after week, relationship after relationship for her man to show that he's attracted to her, pursuant some Harlequin trope about men unwrapping their woman like a gift. That might work once, or once in a while, but by and large she's just telling him (the animal him that runs the show) that she's infertile, or worse, that she's rejecting him. By here one thing should be clear: As a woman, there is no grander judgment you can make than who you choose to sleep with. The men who tell you it's no big deal are just trying to get you in bed. When you sleep with the jerk, and friend-zone the good friend, you are stating clearly and to the world that you value the jerk more than you value the friend--no matter how many ways you try to say otherwise. You are declaring--at the animal, emotional level--nothing short of that one should live and one should die. Your friend (if your friend finds you attractive and sees you choose the jerk) will hear that, and so will the jerk. Again, it's up to you what you do, but just know this is what you do. (A companion essay to this one would explain why women pick the jerk. It's a valid choice, and a long topic.) This, finally, leads to a litmus test you can use, a sort of Rosetta Stone, for translating to men's ears. If you want to know how something about sex sounds to a man, just replace sex with talking (it is a very imperfect analogy, but nonetheless a useful device): Having a girlfriend who's shy about showing off her body is like having a boyfriend who's too shy to talk to you. Saying that you are not, and should not be, obligated to have sex with your boyfriend will feel to him the same as if he declared to you that he is not, and should not be, obligated to talk to you. (Both are true...) Resenting his desire to have sex daily (or whatever) feels to him like it would feel to you if he resented your desire to talk to him daily. You don't want your boyfriend to talk to you because he feels obligated to. You don't want him to resent your desire or need to talk. What you want (most likely...) is for him to want to talk to you, to enjoy talking to you, even if sometimes just because he cares about you and enjoys giving you what you want or need. Maybe he just needs to truly understand what you want or need. (But that is an essay for another day, by another person.) The bottom line is not just that men think about sex a lot, but that sex for men--with someone they find attractive--is the emotional metric for whether they are doing ok in life. This is not by choice, and it cannot be chosen otherwise. It can lift men up, or tear them down. It makes men both predictable, and dangerous. Don't underestimate its priority for men--it is above safety and reason, let alone cultural etiquette. (Some will simply ignore your choice or preference, because that is also an evolutionarily viable strategy--high risk, high reward.) Internalize this, and use it wisely. It can be equally a tool of manipulation, of kindness, or of cruelty. I hope you'll tend toward the middle. <blockquote> "If I became a philosopher, if I have so keenly sought this fame for which I'm still waiting, it's all been to seduce women basically." -- Jean-Paul Sartre </blockquote> <center>[<a href="https://sifter.org/~simon/journal/20230521.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20230929.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240412.html">Next</a> | <a href="https://sifter.org/~simon/journal/indexh.html">>></a>] </center> </blockquote> Home Sweat Homehttps://sifter.org/~simon/journal/20240412.htmlHome Sweat Homehttps://sifter.org/~simon/journal/20240412.htmlFri, 12 Apr 2024 00:00:00 GMT<body bgcolor="#ffffff" text="#000000" link="#334444" vlink="#333333" alink="#ffff00" morss_own_score="2.968877361985921" morss_score="8.458851368669915"> <blockquote style="margin: 0 auto; max-width: 800px;" morss_own_score="2.979948013367991" morss_score="154.17560018728102"> <center> [<a href="https://sifter.org/~simon/journal/20240308.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240703.html">Next</a> | <a href="https://sifter.org/~simon/journal/indexh.html">>></a>] <h2>Friday, April 12, 2024</h2> <h4>Home Sweat Home</h4> </center> [Home automation] <a href="https://hackaday.io">Hackaday</a> is running a little contest that seemed a good fit and excuse for me to describe my home automation system, which I've been meaning to do anyway, so here it is. (I don't expect to win and don't even know what the prizes are. More interested in potential collaborators.) <hr> What started with my dishwasher dying evolved into a new home automation library and infrastructure that makes it super quick to create and integrate new devices--including custom hardware. I'll describe here everything that's automated in my home and show how they're all elegantly tied together, from dishwasher controller, thermostat, RO water valve, wifi temperature sensors for freezers and such (all custom hardware costing less than $100 combined) to zwave and Shelly devices, to radon sensors and geiger counter, to controlling the TV and stereo, to voice commands (hosted in-house), automated reminders and so on. The core of the system on the software side is a (python) library that makes it easy to publish and find attributes by name and type. If you have a widget with a couple of switches, a light, and a dial, you simply publish four values to the home-automation network: two read-only booleans (the switches), a read/write boolean (the light), and a read-only real (the dial). By convention, these are named in a path hierarchy, like "widget/switch1", "widget/dial", and so on. The library takes care of all the dynamical aspects. When someone turns the dial, your (driver) code just calls dial.set_value(v), and that's it. If nobody is currently listening to the dial, nothing happens. If one or more tasks anywhere on the network are listening (whether in the same application, or on a different machine entirely), they are notified (once!) that the value has changed. When and if they care to know, they ask for the latest value and get it. This is a little slower in terms of latency (on a scale irrelevant to humans) but wildly more efficient than the usual approaches in terms of total bandwidth, flow management, and all that practical crud that usually makes this stuff a headache. And again, all of this happens in the library--neither end has to worry about it. If you wanted that widget to control your bedroom lamp brightness, the code would look like this--and you could run this code on any machine on the network, regardless of where the code managing the widget or lamp were running (but it could all be in the same process just as easily): <blockquote><pre> lamp = ha.find("master_bedroom/north_lamp") dial = ha.find("widget/dial") async for value in dial: await lamp.send_value(value) </pre></blockquote> That for loop runs forever, quietly waiting for the dial to be turned, and then sets the lamp accordingly. If the dial generates five values in the time it takes the lamp to receive and acknowledge the value, four of them are never even sent over the network (because they went stale before anyone needed them!). And to be clear, the "async for" automatically sets up a subscription to the dial's value stream in the background, and if you exit the for loop, the subscription is closed. Note it doesn't matter here whether the lamp or dial are on zwave, Shelly, custom, or whatever. The above works even if the lamp or dial don't even exist at the time the code is run: the async for will wait for the dial to come online. If you smashed the (zwave, say) widget and replaced it with a new custom device with new code on the driver side, the above loop doesn't even need to be restarted -- it will simply wait while you're changing the devices, and resume when the (new) widget/dial comes online. Of course the interface to the dynamic values is much richer than just async iterators, but it's all pretty straightforward and obvious like that. It would be very easy to add a nice GUI layer for a code-free experience, such as being able to simply visually connect the "widget/dial" port to the "master_bedroom/north_lamp" port via a "wire" or whatever. And that could run as a separate process with no need to modify any of the existing code (wouldn't even require restarting any processes besides the GUI itself). But so far it's been so easy to code anything I've wanted I haven't needed that personally. I did write a "monitor" which is like a hierarchical file browser for the entire HA network. It's less than 1000 lines of python code and provides a curses-based interface which updates in real time as well as letting you set/change any settable values, graph historical plots, and so on. This is a totally generic browser, non-specific to my particular network or devices. Here's a snapshot of mine right now with a few things randomly opened (the ">" on the left show where the hierarchy can be opened). Note this is just an administrative tool, not really intended to be a primary UI, but it does work as one when needed: <a href="https://sifter.org/~simon/journal/ims/20240412/6943751712880632028.png"><img src="https://sifter.org/~simon/journal/ims/20240412/6943751712880632028.png"></a> Again, all of this is updating in real time. If I want to turn on my Yamaha stereo amplifier, I just go to the "power" line and select True. If I go to the "audio_effect" property, a menu with all the available audio effects appears. To make that work, the driver code for the amplifier simply publishes an additional informational dictionary about the audio_effect variable, under the path "yamp/audio_effect/info", which lists the options for that value. (Everywhere you see "info" above, it is the info dict for the parent object.) Even that can update dynamically though that's rarely needed. As with any system like this, you need to write a little wrapper code for each new type of device, but that amounts simply to creating a dynamic variable object for each value you wish to publish, and then reading/writing it as needed. The dynamic vars look roughly the same on either end of the pipe, so the code in the "drivers" looks more or less like the code snippet above, plus whatever is truly specific to the device. (The main difference is just that the owner of a variable "sets" it whereas everyone else "sends" to it. And that the owner can receive "sends", either via callback or await style. Sends can even raise exceptions, asynchronously, and they propagate back over the network to the calling async send. By convention those are end-user friendly exceptions like "You can't set the brightness above 100" which means any high-level GUI can give useful feedback to users about any device on the network. And incidentally the dynamic variables work great within the same process as well and don't needlessly marshal the values or anything like that, let alone call any network code, so it's quite common for a driver to have tasks awaiting its own values and such.) I've written a generic Zwave device wrapper. The "zooz4in1_2" in the monitor screenshot is a zwave device. If I open the "values" group there it would bring up all the other zwave attributes available for that sensor besides the key ones shown there. "Yamp" above is a Yamaha Amplifier with an ethernet port in the back. They publish the protocol and it was fairly easy to write a wrapper for it. Likewise for the LGTV (WebOS) and so on. In general the philosophy is: create a low-level wrapper around the device which exposes the functionality of the device in the most straightforward way possible (that wrapper should have no dependencies on this home-automation project at all--something you could use anywhere), then write another wrapper/bridge that publishes that API as a bunch of dynamic values (or occasionally calls, which are also allowed), and then later and elsewhere write any actual control code strictly in the generic terms of networked dynamic values, as in the trivial (but real and sufficient) code snippet above. The device wrappers are typically in the same process, but the control code is typically separate. This means it's easy to restart/rewrite particular devices or sub-systems at any time without impacting the rest of the system. (E.g., I can launch the monitor any time from any machine, have many running at once, and so on. Any change made from any monitor of course shows up in all the others in real time. Also my main "AI", or automation and scripting module, is just one process that I can tweak and restart freely without, say, having to recycle the zwave network or whatever.) One feature of this overall approach is that it's really easy to make and integrate custom hardware. For instance I made a bunch of temperature sensors simply by jumpering a $4 ESP8266 dev board to an MCP9808, and writing a tiny bit of driver code that waits for simple ascii commands in UDP packets and replies. If it's gets "temp c" it replies (to the sending IP/port) with the current temperature in Celsius. If it gets "leds on" it turns the dev board's LEDs on. That was all I needed to do, and I've cloned that combo multiple times to make wifi freezer sensors, a remote sensor for my home thermostat, and so on. The low-level python driver code just wraps up that simple UDP protocol as a couple of calls to query or set the temp and leds, and the next level wrapper just publishes those as dynamic values on the HA network. It's a lot of pieces but they're all quite simple, and presto: some new values appear in the monitor. This meant it was trivial, for instance, to connect a motion sensor in my bedroom to the LEDs on the remote temperature sensor (two totally unrelated devices) so that when I get up in the middle of the night, two tiny blue LEDs turn on, which is just enough light so I don't kill myself on whatever landmine my kids left for me. (The flow is: zwave motion sensor -> zwave wrapper -> generic motion event (over TCP) -> trivial code that throws a switch for 30 seconds in response to an event -> generic boolean value (over TCP) -> temp sensor wrapper -> UDP -> ESP8266. But that's all set up with a few lines of code for that trivial bit in the middle.) Notably, the high level code (like the monitor, or a wire-up GUI, or a scripting front end) doesn't need to know that the LEDs are part of a temperature sensor, or even that they're LEDs -- they're first and foremost just a boolean value, which can be used anywhere a boolean value can be. The dishwasher and thermostat are similar, just with a bit more logic on the device itself so that they run reasonably independently even if the HA system is offline. (As an aside, I did two revisions of the thermostat--one in a rush with push-on jumpers in a bug box, and then I went ahead and had a board made for version 2 -- five devices fully assembled for under $100 delivered to my door which is pretty crazy! It works great but I would make some changes next time. If anyone wants to collab on a very-affordable wifi/bluetooth thermostat let me know...) Version 1: <a href="https://sifter.org/~simon/journal/ims/20240412/911341712883953620.jpg"><img src="https://sifter.org/~simon/journal/ims/20240412/911341712883953620.jpg"></a> Version 2: <a href="https://sifter.org/~simon/journal/ims/20240412/5276221712883981706.jpg"><img src="https://sifter.org/~simon/journal/ims/20240412/5276221712883981706.jpg"></a> The dishwasher controller: <a href="https://sifter.org/~simon/journal/ims/20240412/1132491712886294448.jpg"><img src="https://sifter.org/~simon/journal/ims/20240412/1132491712886294448.jpg"></a> With the same pattern, I was able to integrate my geiger counter (one of the first Onyx devices) in a couple of hours, which is fun because: With a couple lines of code, any dynamic variable can be tracked over time in a database, and published on the network as a historical stream, which the monitor can plot. Here is my current air quality plot (some traces lag because the Airthings Waves batch by hour): <a href="https://sifter.org/~simon/journal/ims/20240412/7207701712883638931.png"><img src="https://sifter.org/~simon/journal/ims/20240412/7207701712883638931.png"></a> The vertical bands in the Temperatures plot shows where my furnace was on. They are blue when the A/C is on. The dashed red line shows the current heating target temperature. Note the target temp changes gradually in ramps rather than sudden jumps. (It's so easy to script up when you can just write a little python to tweak some dynamic values. The thermostat controller reads a temperature schedule from a simple text file, and updates the target temperature ramps any time that file is changed. So I just edit that text file as needed or swap it for different ones as the seasons change. Plus I have a "bump" feature that lets us bump the temperature up or down when we're chilly or hot, and that offset gets gradually eroded by the scheduled ramps, so we can tweak the temp when we need without screwing up the schedule. That combined with the remote sensor in our bedroom has made our house so much more comfortable than our old stock thermostat, every single day, that it was well worth the time to make.) Inspired by the ease of making new devices, I cobbled together a quick water valve for our RO system so I can turn it on for a given amount of time (here shown with the cover plate off): <a href="https://sifter.org/~simon/journal/ims/20240412/9471711712884193958.jpg"><img src="https://sifter.org/~simon/journal/ims/20240412/9471711712884193958.jpg"></a> Because it's easy to run processes on different machines, I am able to run speech recognition (faster_whisper) on my desktop machine (while most of the HA stuff runs on a small box that doubles as a router/firewall), and to send the results to the command channel (which is also connected to Telegram), so now I can push a microphone button on my phone and say "fill a gallon" and the RO water will turn on just long enough to fill a gallon jug of water. (The valve box shown above just accepts a command to run for a certain number of seconds and then shuts off, so even if the network goes down it will not keep running.) Most of the important HA features are available in a Telegram chat with my house as a menu of buttons (like turning on/off the bedroom lamps, raising/lowering the thermostat heat, etc.) But the speech recognition turns out to be especially good for parameterized things like "set the [tv/stereo] volume to 35" or "run the water for two minutes twenty seconds" (filling my humidifiers, etc). I can of course type the same things to the same Telegram channel. And having all the different devices controlled together is of course great for "scenes" such as, after my toddlers have twiddled all the buttons and dials on the TV and stereo such that nothing is working at all, I can say (verbally) "start Netflix" and the stereo and TV are appropriately re-configured, volumes set, lamps down, and so on. But that's pretty standard home automation stuff. Our current Telegram menu: <a href="https://sifter.org/~simon/journal/ims/20240412/4613091712884945808.png"><img src="https://sifter.org/~simon/journal/ims/20240412/4613091712884945808.png"></a> The Telegram bot gives us a bunch of other commands and controls as well, which are great when we're away (only accessible from our accounts of course). And again, it was very easy to integrate because it works in terms of vanilla dynamic values with no concern for where they're coming from. I also wrote a simple Android app which turns an old Android phone into a remote camera. From the monitor (the same vanilla monitor above) I can select front/back camera, resolution, focus modes, and so on. Any app on the network can fetch an image from any camera at any time, and I use that to make motion-timelapse security videos. Here are the two pointing out the front of the house atm: <a href="https://sifter.org/~simon/journal/ims/20240412/1799641712885337568.png"><img src="https://sifter.org/~simon/journal/ims/20240412/1799641712885337568.png"></a> I think it would be fun to do a timelapse from each camera at exact solar noon every day. Haven't done it yet but it should just be a few lines of code when I get around to it. I have some USB webcams which are published the same way as the android cams, so my full security camera panel is a mixture of source types--which the code that displays them is completely unaware of. Because of the push/pull nature of the library, it's an automatic feature that when I launch the viewer on my laptop or wherever, the driver on the box with the USB camera opens the camera and starts recording. The same thing happens if I simply open the camera hierarchy in the monitor down to the level of the raw jpeg stream (at which point I start seeing a succession of the first few byte of jpeg encoded images in the monitor). When the last consumer closes, so does the camera. Again these are all separate processes on different machines. For anything I want running regularly, I add it to the process manager for that machine, which makes it available as a process in the monitor. This allows me to quit and restart processes (say for a code update) from the monitor without even having to remember which machine they're running on. Also makes it easy to see the running status of every process/driver on the network regardless of where it's hosted. (On my todo list is to make a red/green status panel that just shows at a glance that everything is running as expected, with the ability to drill down into red items to find out what's wrong, restart processes if needed, and so on. But so far everything has been so robust, it all "just works" and I haven't had to worry about this.) If there is interest, I would be happy to open source everything. My sense is it would be in competition with Home Assistant and such, which are so extensive now I doubt it's worth the trouble, which is why I haven't done so already. But there it is. <center>[<a href="https://sifter.org/~simon/journal/20240308.h.html"><<</a> | <a href="https://sifter.org/~simon/journal/20240308.h.html">Prev</a> | <a href="https://sifter.org/~simon/journal/index.html">Index</a> | <a href="https://sifter.org/~simon/journal/20240703.html">Next</a> | <a href="https://sifter.org/~simon/journal/indexh.html">>></a>] </center> </blockquote> <hr> <hr> </body>