Rewiring

cable-wire-recycling-AZZ11.jpg

At least 10-12 inches of snow are falling outside the window, and I spent the evening moving between various states of slipping in snow, to sliding in cars, to overheating indoors. And throughout these transitions of state, the term “rewiring” is flickering through my thoughts.

The venerable online etymological dictionary has an entry for “wire” that states,

Old English wir “metal drawn out into a fine thread,” from Proto-Germanic *wira- (source also of Old Norse viravirka “filigree work,” Swedish vira “to twist,” Old High German wiara “fine gold work”), from PIE *wei- (1) “to turn, twist, plait” (source also of Old Irish fiar, Welsh gwyr “bent, crooked;” Latin viere “to bend, twist,” viriæ “bracelets,” of Celtic origin).

Like working glass from a rod to an ornate and beautiful capture of heat and time, these definitions suggest that modest wire is a tangible expression of effort, or, a platted braid of circumstance (think of hanging cords, twisted around from radiating vents or curious cats, turned into something resembling the industrious form of wire).

Wire is dangerous. Wire is necessary. And wire is precise. This is no hip-shooting, boat wrangling rope rodeo. No, wire is the conduit of the 21st century’s most precision modes of communication, that by which the finest of movements are illicited from airplane rudders, and much more. Wire is a forgotten pillar of our modern infrastructure, with roots back to aesthetically pleasing arrangements of precious metals.

Rewiring, then, must be a reversal of some precise or circumstantial expression. At least temporarily to make way for a new pattern. Rewiring is slow. Rewiring is understanding potentially complex networks. And rewiring is permanent. It’s rare that we rewire a house, pull out the old, install the new, and ever rerewire the house with the old. Sure, importantly, that network of old wire can be reused, repurposed, or revered, but it’s unlikely it will ever power that same rotary dialer again.

And so, “rewiring” represents to me evolution. Irreversible change. Which isn’t always a bad thing.

False Conditions

Was reminded this morning of a lesson that drifts in and out of working on systems with lots of moving parts: all improvements are inextricably based to the current condition of supporting infrastructure.

Said another way: anything you do, anything you change, is probably based on information available to you at the time.

But this isn’t the lesson. The lesson is that that kind of decision making is often flawed. I’m sure this is of no surprise to many, but I’m uncomfortable how each iteration of an improvement to a particular part of the system brings this same lesson home. Fool me once, shame on you, fool me twice, yada yada.

A concrete example might help.

We have series of pipes and routes in our server-side API that abstracts routes for images from our IIIF-based Loris image server. So, we can ask for http://foo.bar/item/goober:tronic/thumbnail and get back a thumbnail at the more complicated URL path, http://foo.bar/loris/fedora|goober:tronic/full/full/0/default.png. The latter is not semantically meaningful to many, and contains hardcoded infrastructure such as loris in the URL. Our image server may change, and our goal is to have Cool URIs for things like thumbnails, metadata, etc. As always on this blog, over-simplification for the sake of idea exploring.

Recently, we had the rare and supremely delicious surprise of server kernel patching improving server-side rendering of images in our python-based image server, Loris. Dramatically. We are still exploring precisely what explains the speed increase (perhaps fodder for another post), but suffice it to say, it’s great. However, thumbnails started to break. The reason, one of our image proxies was streaming the results with the requests python library. When speeds / rendering / IO was slower on the server, this sped up the load time for thumbnails. But when the server speed increased, it revealed what I’m assuming was some kind of race-condition as the bits jumped through these proxied hoopes. Again, this is all speculation at this point, but the fact remains that removing the streaming flag from a particular request has fixed the problem, and, the thumbnails load even faster.

Our original design to stream a response sped up the load-time with a particular set of server conditions. Now that the conditions have changed, that decision is no longer correct. How interesting, that a decision once correct, becomes flawed over the passage of time. Such is a day in the life of managing a system with lots of moving parts.

Strange Attractors Part 1

A post, in two parts.

Part 1: Strange Attractors, and their strangely attractive backstory

I recently had the pleasure of driving to Texas and back, and while on the road, partook in some podcasts. One of them was called, “Stuff You Should Know”, and dedicated an episode to Chaos Theory. They did an admirable job with a slippery idea, and while I took away some new insights, this topic was not new to me.

No, I’ve been a fan / advocate / evagnelist / lunatic / devotee of Chaos Theory since the heady days of 2001. Early in my undergraduate forays into math and physics, I convinced a professor to let me explore the interesting ramifications of nonlinear and fractal geometry. Before going any further, if you’ve searched for “fractal geometry” or even just “fractals” on the internet, you’ve gotten some websites that looked like they emerged from the 1990’s. You see, the people that love fractals have a need to share their insights. I was firmly in this camp.

I graphed henons in QBASIC, porting code from the appendix of – gasp – print books. I drew sierpinski triangles everywhere. I used “bifurcate” every chance I could. It was a wild high, and I chased that tempestuous beast into the darkest of nights.

I became fascinated by coastlines: did you know they are infinitely long? On a map, the circuitous circumference around an island nation like Iceland may look to be… ~1,000 miles? But in reality, we can dive into any fjord or inlet, crouch low and begin poking around in the rocks that roll gently in the lapping tides. Where is the coastline? Is it before or after that pebble? If that pebble is included, don’t we have to include it in the total distance of the island perimeter? Yes, you most certianly do, as a responsible member of society, a reasonable steward of information.

This leaves us with an uncomfortable truth that “edges” and “boundaries” are often much more complex than we anticipated. Also revealed from fractal geometery, the same complex gravitational forces that create a valley of flour in your bowl when an egg is cracked into it, are the same that shape the timeless Grand Canyon (at least, many are shared - in all likelihood, your flour valleys have not been whipped by wind and rain for millenia). But the point remains: brocoli looks like trees, and a bunch of geese flying look suspiciously like reflections of light on lazily undulating waves.

And yet there is so much more to this story, which again, I must commend the hosts of that podcate for nobly wading through, with patience and a singular sense of where they are in the discussion. Restating known things can be useful for honing intuition, and setting the stage. So to explore the strange attraction of Strange Attractors, we must go back to Sir Isaac Newton, King Oscar II of Sweden, and the “n-body-problem”.

Newton could predict apples falling, and was having good success predicting comets and cannonballs. So what about predicting the location of a handful of planets 10, 100, 1000 years out? Not so much:

“Knowing three orbital positions of a planet’s orbit – positions obtained by Sir Isaac Newton (1643-1727) from astronomer John Flamsteed[6] – Newton was able to produce an equation by straightforward analytical geometry, to predict a planet’s motion; i.e., to give its orbital properties: position, orbital diameter, period and orbital velocity.[7] Having done so, he and others soon discovered over the course of a few years, those equations of motion did not predict some orbits very well or even correctly.[8] Newton realized it was because gravitational interactive forces amongst all the planets was affecting all their orbits.”

And here, the badlands of Chaos Theory and nonlinear dynamics peeked forth. Scientists and mathematicians started to understand that the gravitational pull from each planet was simultanesouly pulling on one another. Each moment, or “iteration”, magnified any variances or inaccuracies in the initial measurements that were plugged into the equations. If a cue ball hitting three pool balls with speed x is turned into equations, we can predict where they will be after 1000 “moments”. However, if speed x is ever so slightly different, those pool balls will be in wildly different places just a few “moments” later. This is a gross over-simplification of the n-body-problem, but it gets at the heart of it.

King Oscar II held a contest for anyone who could solve this problem. As I didn’t intend to delve too deeply into the history, but instead muse on what it might mean in the present, I continue to muddy and gloss over the finer points of this. This article explores the contest and solution in much better detail. But for our purposes here, Poincaré pointed out that it was unsolvable, and won the prize. He proved that a perfect predition relied on infinitely accurate measurements, and we know that’s not possible. And Chaos Theory was born.

Fast forward lots of years, and we’re finally getting back to Strange Attractors, and the meteorologist Edward Lorenz stumbld on this very problem while trying to distill complex equations around thermal dynamics to a simple form. This passage from Wikipedia sums it up nicely,

“Minute variations in the initial values of variables in his twelve-variable computer weather model (c. 1960, running on an LGP-30 desk computer) would result in grossly divergent weather patterns.[2] This sensitive dependence on initial conditions came to be known as the butterfly effect (it also meant that weather predictions from more than about a week out are generally fairly inaccurate).[13]”

And here is what he graphed:

The hands grow cold, the coffee cup has tipped past halfway. Where are we in this discussion? Those dark spots in the graph above, those are the strange attractors we’ve danced around. Why does this matter? Why is interesting? How is this related to coastlines, pebbles, brocoli, and eggs? That, is fodder for another post…