Recently in chaos Category

Author C.G. Masi's forthcoming novel looks at how technology developers go about their business in a corporate environment.

Many thanks to the loyal readers of this blog, who have put up with a low posting frequency over the past few months. My excuse is that I've been trying to get my next book into production. It's nearly there, so I should be able to provide more frequent posts to this blog.

Readers who enjoy my commentaries on how technological advances affect current events will have a lot to interest them in the book, which should be in bookstores around mid-summer. Entitled Red, it is a novel whose main characters work in a private applied-physics research company. The title comes from the nickname for the central character, Judith McKenna, who is a tall, athletic, young mathematician, who tosses everything away to search for her missing father after the authorities have exhausted all conventional means of finding him. Her faltering quest is saved by Doc, her mentor and sometime lover, who shows her how to organize the scientific and technical resources she didn't even realize were available to solve the mystery.

To reach her goal, she needs to learn techniques of organization, resource allocation, team building, and decision making under uncertain conditions. If you thought such issues were dry and academic, it's because you haven't seen them played out in the emotionally charged, risk-filled environments where real-life technology developers live and work, where millions of dollars, careers, and even lives are often at stake, and any mistake can lead to disaster.

If you think that's hyperbole, take a look at what's happening right now in the Gulf of Mexico.

We're now doing the final polish edit on Red. The schedule calls for that to be done before the end of June, at which time the book will go directly into production.

Most of the work is now in the hands of others, so I will have more time to devote to looking at how technology interacts with society, which is the focus of this blog. I plan to start by sorting through the issues surrounding the Gulf oil disaster. What actually happened? Who should really be pointing fingers at whom? Are the actions contemplated by the Obama Administration likely to help the situation, or make it worse?

Hopefully, I can help make sense of it all.

Graphing historic DJIA prices on a semi-log plot shows that our financial markets experience a price wave with a 20 year period superposed on a steady long-term growth trend.

Yesterday was the first day that the Dow Jones Industrial Average (DJIA) managed to close above 11,000 in a long time. It had been flirting with that level for almost a week, now, and had crossed that level several times intraday, but never held it through the close of trading.

The media, of course, made a significant bit of noise about it - enough that my wife asked me, after reading the headlines in the local newspaper, whether it really was a good thing. Now, my lady is quite bright (she's working on her second Master's degree), but, as a humanities major, her long suit is not the kind of quantitative analysis necessary to interpret what moves in various economic metrics, such as the DJIA, mean to actual people trying to get by.

"Is the Dow over 11,000 a good thing?" she asked.

"Yes, but it doesn't really mean much," I replied. "I predicted it'd spike over 11,000 a month or so ago, then slide back. But, things are pretty much on track."

Analysis I did last fall (see image above) indicates that the DJIA is just about exactly on its long-term track. It should be just peeking above 10,000 right about now. Since we've just experienced a short spike down (You do remember we've experienced a recession over the last year and a half, don't you?). We can expect an overshoot on the recovery, then settling back to the long term trend modified by a chaotic wave with a period of about 20 years.

In the future, we can expect to see a slow rise with a long term trend of zero to a few percent for about the next five years. The trend should steepen thereafter, reaching a maximum about 2020. In the meantime, expect the DJIA to be in a trading range between 9,000 and 11,000.

The important thing to understand is that the huge price swings that many of us capitalized on over the past 18 months are unlikely to repeat, barring exigencies. Since stock traders make money by cleverly exploiting stock volatility, they won't do quite as well as they have over the past 20 years. Expect the real money to be made by investing in dividend-paying stocks. Expect portfolio returns in the 5-15% per annum range to be the norm. A good model for this investing environment would be the rather boring period from about 1965 through 1980, when the DJIA stayed essentially flat, with only short term ups, and downs.

Sorry, folks.

Signs of global warming

A flurry (pun intended) of articles in today's issue of The Wall Street Journal prompted me to drop another post about the controversy surrounding climate change research and efforts to curb global warming. Readers who have followed my posts here and in the Ask Charlie blog I wrote for Control Engineering know that I'm no fan of the IPCC report upon which most of the current nonsense is based. It's not that I think that there's anything wrong with the basic thesis that dumping loads of carbon dioxide into the atmosphere will likely ratchet up global temperatures, my problem is that so much of the so-called research, and especially the conclusions drawn therefrom, are prima facie so much politically motivated dreck (or to use the proper Yiddish spelling drek).

As I see it, there are two basic problems. First, the conclusions are based on a sophmoric physical model. Second, who ever said that higher global temperatures would be a bad thing, anyway?

The theory of global warming is based on a simple physical model - the greenhouse model - which is, in turn, based on the solid physics of radiative heat transfer. Specifically, it starts with the observation that the opacity of most atmospheric gasses is wavelength dependent. That is, while most of these gasses appear transparent to visible light, they are more opaque (sometimes very opaque) to infrared wavelengths.

So, the radiative power flux of sunlight, a large fraction of which comes at visible wavelengths, gets through the atmosphere to warm the ground. The warm ground tries to radiate that power back out at lower wavelengths (basically, the color temperature of sunlight is about 6,000 K, while that of radiation from the ground is about 300 K). The infrared, however, is absorbed by the dense lower atmosphere. Ergo, the ground and lower atmosphere, which are roughly in thermal equilibrium, get warmer. Increasing the density of the more infrared-absorbtive gasses, especially carbon dioxide, (so the theory goes) will necessarily increase the infrared absorbtion, and lead to higher temperatures.

We teach this model as an example in second-semester freshman physics. It's simple, easy to understand, and illustrates the mathematics of radiative heat transfer (which is what we're trying to do in freshman physics). The only problem is that the model is dead wrong. The real world is vastly more complicated. The difference is so extreme that any conclusions drawn from the greenhouse model are unlikely to correspond to anything in the real world.

One of the biggest problems is that meteorologists have known for decades that the weather system is chaotic. Weather patterns cannot be reliably predicted for a time scale longer than about a week. Weather, of course, is critical to radiative heat transfer, so asking a climate model that uses radiative heat transfer to predict anything beyond about a week is simply stupid. Other parts of the climate system are similarly chaotic, such as solar flux variability, making the prediction of future climate via computer models an exercise in futility. It is of academic interest, but of academic interest only.

Moving on to the second problem, who says global warming is a bad thing, anyway? The medieval warm period (look it up) ushered in an age of prosperity, cultural advancement, and generally really good times. It was followed by the the Little Ice Age, which brought with it famine, plague, and death. Who th' heck wants that?

Lessons from history, and prehistory uniformly lead to the syllogism:

cooler = bad;

warmer = good.

You do the math.

Business cycles are driven by a macroeconomic feedback mechanism that has a multi-year cycle time. Employment is one of the last economic metrics to show recovery because the process starts with unmet demand for goods and services, and only ends with jobs.

In every economic downturn, Chicken-Little pundits squawk about how we can't have a sustainable recovery until employment figures show improvement. Any investor, and here I use the word "investor" in its broadest sense to include those who put resources to work, not just those who invest in stocks and bonds, who listens to this drivel is destined to fail, and fail disasterously.

Macroeconomics - the study of large-scale economic trends affecting an economy as a whole - is based feedback loops that drive business activity. These loops describe causal relationships between economic factors affecting business. For example, an increase in production levels generally pushes employment up. Each of these causal relationships involve a time delay. So, when production levels increase, especially from a depressed level, employment does not rise until production levels exceed capacity at the current employment level. This takes time, as does the process of hiring new employees.

These delays are what cause business cycles in the first place. If we use, say, buggywhip manufacture as a hypothetical example, we might say that it takes 18 months for the buggywhip business to respond to a sudden change in the overall demand for buggywhips. So, if New York City should pass a law banning motorized vehicles, so all the Yellow Cabs in the city had to be replaced by horse-drawn surries overnight, that would ratchet up demand for buggywhips. Because it takes 18 months for buggywhip manufacturers to respond, actual sales of buggywhips would not stabilize at a level reflecting the new demand until a year and a half later.

Business cycles occur because it is not possible for businesses to precisely meet demand. In the buggywhip example, assume that there are two buggywhip manufacturers in business at the time the New York law passes. They will both attempt to grab more than their fair share of the enormous new market. Part of driving sales is assuring customers that you can actually deliver the goods ordered. So, both manufacturers will expand production faster than necessary to just meet demand. In addition, during that first 18 months, it will be clear that the established manufacturers won't be able to meet demand. Outside entrepreneurs will see this as an opportunity to jump in to the expanding market, by starting rival buggywhip manufacturing operations.

The result is that some 18 months after the new law passes, worldwide buggywhip manufacturing capacity will greatly exceed demand. Inventories of unsold buggywhips will expand. Buggywhip prices will fall. Marginal buggywhip manufacturers will fail. Buggywhip production capacity will drop. By three years into the process, we'd be back to having inadequate production capacity to meet demand, and the whole thing would start over again.

Boom and bust cycles like that are not some aberration, or the result of faulty business strategies, or some market inefficiency that politicians can erase by passing laws, it's how things inevitably work. In fact, most complex systems, such as economies, consist of multiple such cycles that operate on multiple time scales. Basically, they're all chaotic systems, which is why long term charts of practically every economic indicator - from long-term jobs trends to prices for individual stocks - look like profiles of the Andes Mountains. They're all fractals, which is the pattern most often associated with chaotic systems.

Economic expansions, recessions, depressions, and recoveries are actually just business-cycle components. As any Taoist sage could tell you, whenever the economy is expanding, you know that a contraction is on its way. Similarly, a depression always presages a recovery. It's inevitable. The Great Depression of the 1930s was, when looked at from a longer perspective, just a particularly deep bottom of the overall business cycle. The huge expansion we experienced during the 1990s was, conversely, a particularly robust phase of the overall business cycle.

This latest contraction, which started about 2005, and will probably not completely play out until 2015, was another particularly nasty dip in the more or less regular cycle. It's as inevitable as the tide.

So, getting back to jobs data, and the usual panicky predictions of a so-called "jobless recovery," the reason employment data have not significantly improved is that it's just too early in the process for it to show up. Those who ask: "How can sales recover when employment is down?" simply don't understand how the business cycle works. Sales aren't driven by jobs, it's the other way around, with a significant time lag between.

Jobs are driven by production requirements. As any industrial engineer could tell you, production is driven by inventories, not by demand. Demand is an intangible that is very difficult to predict or measure. Inventory levels, on the other hand, are easily measured and better reflect a company's ability to sell the products it makes.

In the real business world, the first thing to recover after a recession is demand. It begins to recover when end users have had their belts cinched so tight for so long, that they have no choice but to by new stuff. Demand for food starts to rise, for example, when pantries start to look bare. It makes no difference whether the family bread-winner has a job or not, when there's nothing for dinner, somebody makes a run to the store. Even if you have to beg a cup of sugar from the neighbors, that sends the neighbors off to the store for more sugar, increasing the demand for sugar. Therein lies the disconnect between jobs and demand.

Demand seems to have hit bottom about six months ago. Since then, we've been working off inventory that built up at the start of the downturn, when production still exceeded demand. Next, production has to rise (pulled by further increases in demand) until it exceeds capacity at the present depressed employment levels. Only then will employment figures begin to rise.

Don't look for employment metrics to turn up until at least the end of the first quarter 2010. The reason it hasn't happened yet is that it's just too darn early.

The 11 year solar magnetic cycle is associated with the natural waxing and waning of solar activity. On longer time scales, the sun has shown considerable variability, including the long Maunder Minimum when almost no sunspots were observed, the less severe Dalton Minimum, and increased sunspot activity during the last fifty years, known as the Modern Maximum. Source: Wikipedia. This figure was prepared by Robert A. Rohde and is part of the Global Warming Art project.

During the 1970s, I conducted an (unpublished) meta-analysis of data Charles Greeley Abbot collected from various sources in the early 20th Century to look for cross correlations between his solar irradiance measurements, sunspot index measurements, and weather patterns in various cities. The meta-analysis showed a significant positive correlation between solar irradiance and sunspot data, and a partial correlation between them and the temperature data.

Abbot, like nearly all astronomers and astrophysicists of his time, firmly believed in a negative correlation between sunspot index and solar irradiance, rather than the positive one his data showed. He noted the partial correlation between sunspot index and temperatures, but his prejudice about the correlation between index and irradiance led him to reject the effect as spurious.

By the end of the 1980s, the positive correlation between solar irradiance variations and sunspot index variations had been confirmed by satellite measurements, overturning astrophysicists' previous view. This allowed partial explanation of historically observed climatic variations, specifically the so-called "Little Ice Age" in the latter half of the second millennium, by reduction of solar activity observed through anomalies in the sunspot index, specifically the Sporer, Maunder, and Dalton minima. This research strongly indicates that solar variability is also an important input to the climate system that is certainly not under human control.

Now, it is becoming clear that the climate system is highly complex, with multiple positive and negative feedback loops, as well as a large number of independent forcing inputs, only a few of which are under human control (see "Aerosols Cloud Climate Picture," Science News, v. 176, n. 11., pp. 5-6 for a brief synopsis). These are characteristics of a chaotic system

Paleontologists and geologists have pieced together a fairly complete, though not necessarily detailed, picture of Earth's climate over the 4.5 billion years of the planet's existence. This picture shows a chaotic climate capable of varying over a wide temperature range. On short time scales, weather patterns are now acknowledged to be chaotic, with a horizon of predictability on the order of a week.

Taken together, these bits of information lead one to the conclusion that Earth's climate exhibits chaotic behavior on all time scales. It is, basically, a chaotic system.

Now, let's look at efforts to control climate change. We are attempting to use a chaotic system (global politics) to harness a second chaotic system (social, economic, and technical institutions) to control a third chaotic system (Earth's climate), when not all the forcing variables (e.g., solar irradiance, geology) are in our hands, anyway.

This sounds like a fool's errand.

I suggest that we could much more effectively apply our energies to developing means to react to climate change that is inevitable, than to the fool's errand of trying to direct it. Climate change, in any direction, has both positive and negative affects. It would be far better to direct our efforts toward engineering social systems, laws, and technologies to take advantage of the positive effects, and ameliorate the negative effects.

The Mandelbrot set is the most famous illustration of a fractal in two dimensions. The same mathematics can be used in one dimension to describe financial market price activity. Source: Wikipedia

The other day I caught part of an interview with a spokesperson for Elliott Wave International on Bloomberg TV. Elliott Wave International is an organization devoted to promoting the use of a market-analysis technique developed in the 1930s by Ralph Nelson Elliott, called the wave principle. The most important thing to come out of the interview, to my mind, was an off-hand comment by the spokesperson that stock-price movement "is a fractal."

A fractal is a mathematical waveform with a texture that is of the same form at all spatial-frequency scales. For example, we've all seen those neat illustrations hung in hotel lobbies and meeting rooms that include what appear to be strikingly realistic silhouettes of mountain ranges. Artists make those illustrations by simply tearing a piece of construction paper, and airbrushing past the torn edge. Since construction-paper tears and mountain-range silhouettes are both fractals, the torn paper edge bears an uncanny resemblance to a generic mountain range.

The most famous fractal illustration, and the one mathematicians use most often to illustrate the properties of fractals, is the Mandelbrot set. I think people like to use it as an illustration because it's amazingly beautiful. It also has the advantage of clearly illustrating fractal properties. Unfortunately, it has no practical value, so using it to illustrate fractals leaves the impression that fractals have no practical value.

Nothing could be further from the truth.

Like all fractals, the Mandelbrot set repeats itself on all length scales. It thus displays the property of self similarity. Similarly, if you take a typical stock-price chart covering a time span of, say, one month, and compare it to an intraday chart, the two will look remarkably similar, and will also bear a striking resemblance to the mountain-range silhouette, and the torn paper. They're all fractals.

Self-similar charts, such as stock price charts, all have two spatial frequency properties: they're combinations of "tones" at a wide range of spatial frequencies, and the amplitude of any one tone is inversely proportional to its spatial frequency. Tones at lower spatial frequencies - so they take up more of the chart horizontally - also have proportionally higher amplitudes - so they take up more of the chart vertically.

Comparison of stock-price charts covering a wide range of time spans shows that they also show self similarity, and are therefore fractals. Financial markets exhibit this behavior because they are chaotic systems. That is, there are a large number of independent actors making buy/sell decisions based on different interpretations of the same enormous, but incomplete, data set. Each actor moves the price of an individual financial instrument slightly, but none has enough pricing power to exercise significant control alone. Everyone has an effect, but nobody has a controlling effect.

That is what drives fundamental stock analysts bonkers. Each property of the underlying company, each news item about the company, each pronouncement by a politician, has an affect - often a noticeable effect - but none of them has a determining effect. There's just too much else going on that also has an effect.

It is important to recognize that there are varying degrees of self-similarity. The Mandelbrot set displays the strongest type of self-similarity (exact self-similarity), meaning that it appears exactly the same at all spatial scales. Stock prices, on the other hand, display the weakest type of self-similarity (statistical self-similarity), meaning that the figure has numerical or statistical measures that are preserved over different spatial scales. The fact that stock charts have only statistical self-similarity is what makes stock-price movements inherently unpredictable.

Because Elliott wave analysis starts from the observation that charts of financial-market prices are fractals, the wave principle, out of all technical analysis tools, has the best hope of making sense of financial-market price movements. Elliott's original work was hampered by the fact that understanding of fractals was limited in the 1930s, and understanding of chaotic systems was effectively nil until the 1960s. Some misunderstandings, such as the representation of waves as triangular, rather than sinusoidal, still persist, but anyone interested in learning technical analysis could do a lot worse than beginning with Elliott wave analysis.

Figure 1: Financial markets exhibit variations at all time scales with amplitude proportional to period. Based on data from stockcharts.com.

Yesterday kicked off the first day of the NIWeek 2009. This is the annual National Instruments user group meeting. NI to followers of this company, makes hardware and software for data acquisition and control applications. For general engineering and research laboratory software, the company's LabVIEW product enjoys a dominant position. The way the company has built and maintains this position is by working hard to anticipate major technology trends that will affect their users, and what their executives learn from this ongoing exercise comes out in comments during keynote speeches at NIWeek.

During NI Chairman James Truchard's opening comments this year, he expressed the opinion that some of the economic difficulties we have experienced globally over the past year can be traced to inappropriate use of mathematics in finance

"... Mathematics, the way it has been taught in business schools the last few years," he said, "has a serious problem ..."

Dr T, as he is known to his friends in the industry, also trotted out the oft-quoted comment by Warren Buffet: "It seems like the higher mathematics, with more false precision, should help you, but it doesn't."

This reminded me of a couple of themes I've been studying over the past few years chaos and fuzzy logic as they apply to things in general, and financial markets in particular. These are surprisingly easy to understand, easy to explain, and universal in application.

Chaos has bad connotations in western philosophy, which guides most of what we do for better or worse. In western thought, chaos is the enemy of order, and one of western civilization's stated goals is maintaining order. Eastern philosophy, on the other hand, seeks to balance the "forces" of yin and yang. These are complex concepts, but the important point for this discussion is that, among other things, yin is an order-seeking force, and yang is a chaos-seeking force. The theory is that both yin and yang are necessary for good health, and specifically must be maintained in an dynamic balance.

Mathematically, chaotic systems can most easily recognized by being in constant motion with cycles represented at all time scales, and the amplitudes of these motions are roughly proportional to their periods. Thus, weekly fluctuations tend to have an amplitude 5-10 times larger than daily fluctuations, with the largest amplitude movements appearing on decades-long time scales.

What makes chaos important is that it can be shown to govern the operation of virtually every complex system, from ocean waves to all human history. If it's a large dynamic system with lots of independently moving parts, its behavior most likely is chaotic.

Fuzzy logic, on the other hand, seems unmathematical for a different reason. The term sounds like a synonym for "fuzzy thinking," which people associate with foolishness and ignorance. Mathematically, however, fuzzy logic is a way of dealing logically with things that cannot be precisely quantified.

Humans, in fact, make nearly all personal decisions - especially life-and-death decisions - using fuzzy logic.

When you merge your car into a busy traffic lane, for example, you don't get out a radar gun to precisely measure the speeds of vehicles already in the lane, then analyze video images of the spacings between cars, then do a calculation based on the measured data plus known performance of your vehicle at various settings of throttle and steering controls. You don't have the equipment available to make the measurements, and you wouldn't have time available to make the calculations if you did.

Instead, your brain converts sense impressions and memories from previous encounters into more-or-less broad categories. Spaces between other cars, for example, are "tight," "okay," or "really big." Their speeds are "slow," "medium," or "fast." Your car "accelerates fast," "moves out okay," or "handles like a dead whale." Instead of a dozens of variables with 12-bit precision, your brain reduces the data set to a handful of variables with at most two-bit precision. It also doesn't use arithmetic to work out the answer, but a fuzzy logic algorithm that resembles a decision tree. The answer comes back as "it's safe - go for it;" "gee, it's borderline," or "that looks like a trip to the emergency ward." The final fuzzy calculation says "no" in the last instance, causes you to look further in the middle instance, and spurs you into action in the first instance.

That's fuzzy logic. It's the fastest and most reliable way to make difficult decisions when time is limited (as it always is). It's also virtually the only way to deal with chaotic systems, which, as I said above, almost all real systems are.

Probably (in a fuzzy-logic sense) the way to solve the problem Truchard and Buffet identified, is by mentally taking chaos and fuzzy logic out of the "advanced mathematics" cubbyhole, and teach it to high-school freshman. If we did that, we'd quickly have a generation with the mental tools needed to analyze and deal with the real world, rather than a hypothetical world of mathematical abstractions.

As with so many terms bandied about in mass media, "Smart Grid" is a cutesy umbrella term that allows politicians, analysts, and newscasters to vaguely refer to a collection of technologies that neither they nor their audiences fully comprehend, with advantages that are easily stated, and of uncertain measurability.

While that sounds pretty negative, let me point out that nothing in the above paragraph says anything against the technologies themselves, or their value, but merely pans vague marketspeak terms in general, and the folks who rely on them for ... anything.

Smart grids are part of a general technology trend toward incorporating embedded microcontrollers and data-communication capabilities into all sorts of previously existing devices. For those unfamiliar with them, a "microcontroller" is an integrated circuit that includes a microprocessor and peripheral circuits that allow the microprocessor to sense conditions and events in the external world (data acquisition) and put out signals to drive actuators in the external world (control).

Perhaps the first "smart" devices were automobile engines, which came under microprocessor control during the late 1970s, long before the term "smart xxx" became current. Such engine control modules (ECMs) sensed such variables as outside air temperature and throttle position, and used that information to control such parameters as fuel/air ratio and spark timing. Later, ECMs gained the ability to communicate with additional embedded microcontrollers managing such functions as anti-lock braking systems (ABS) and alarm systems. Modern automobiles now contain dozens of networked microcontrollers operating nearly all functions.

Today, most significant appliances operate under guidance of microcontrollers. Microwave ovens, dishwashers, clothes dryers, televisions, and home thermostats are familiar examples. The extent to which manufacturing operations rely on "smart" technology is even more profound.

Electricity generation and distribution networks, however, are far behind other industries in incorporating smart technology. That is the impetus behind all of the noise and fury about "Smart Grids" in the media.

To be fair, there are significant barriers to incorporating smart technology into electric-power infrastructure. Most significantly, it is imperative to keep the system operating reliably while applying new technology to it. Second, the cost of upgrading existing equipment that was never intended to be part of a computer-integrated system is, shall we say, large. There are many additional issues to be considered when making the move to smart utility grids.

The motivation to incorporate computer control and networking technology into the electric power system is not just to make it more "modern." The concept avoids Scheiber's Rule (Just because you can doesn't mean you should.) by solving a number of present and future problems arising from electric-utility development trends.

The first issue is the fact that the present distribution grid developed from early systems where a single generating plant distributed power to an isolated netword of loads. That placed the responsibility for maintaining voltage, frequency, and phase of the provided electricity squarely on one generating facility. Such installations are amenable to simple closed-loop control.

Later, but still quite some time ago, outputs from multiple generating plants were combined to supply power to the user network. That created the issue of coordinating the output levels and phases of the sources. At least, the sources on a given network were controlled by a common authority capable of centrally guiding the generators via more complex closed-loop control.

Problems became serious when power-distribution networks were interconnected to allow power sharing between sources operated by separate authorities. This makes simple reactive closed-loop control problematic. When you have multiple agents independently providing control inputs in response to observed conditions, the system becomes chaotic. This is not a slam on the engineers who designed and operated the system. It's a fact of life dictated by mathematics. Voltage variations, unpredictable frequency and phase shifts, and seemingly random catastrophic failures ensue.

Happily, all the folks on the supply side of the system were highly intelligent professionals who realized that the only solution was to co-operate their power-generation controls. We'll call it meta-control, where individual operators don't blindly react to every movement of the controlled system, which is what drives the system into chaotic behavior. Instead, when they observe a departure from nominal status, they first communicate among themselves, and devise a coordinated response that brings the entire system back toward nominal.

You can do that when there are relatively few operators. As the number of operators grows, the time needed to communicate and devise a coordinated strategy becomes longer, while the frequency and severity of divergences become more severe.

In the past, the economics of power-generation have favored large generating stations because they can be made more efficient. Costs for fossil fuels and nuclear power scale more slowly than generating plants' output. Emerging energy sources, such as photoelectric and wind power, have been billed as "free energy sources," although they are nothing of the kind, so power-plant efficiency figures less in the installation decision. Thus, we expect to see many more smaller plants. With more small plants, the number of sources that need to be coordinated will rise dramatically, and system-control cost and difficulty will increase.

The assumption is that increased deployment of smart-grid technology will make it possible to maintain system control in the face of increased chaos. High-speed data sharing is to improve coordination while expanded computer automation improves the speed and quality of meta-control decision making.

According to Wikipedia, support for smart grids became federal policy with passage of the Energy Independence and Security Act of 2007. The law, Title13, set out $100 million per fiscal year in funding for fiscal years 2008-2012, established a matching program for states, utilities and consumers to build smart grid capabilities, and created a Grid Modernization Commission to assess the benefits of demand response, and recommend protocol standards.

The Act directs the National Institute of Standards and Technology (NIST) to coordinate the development of smart grid standards, which the Federal Energy Regulatory Commission (FERC) would then promulgate through official rulemakings. Smart grids received further support with the passage of the American Recovery and Reinvestment Act of 2009, which set aside $11 billion for the creation of a smart grid.

Progress has been swift, as it needs to be. Federal Energy Regulatory Commission (FERC) issued a proposed policy statement and action plan on 19 March 2009 for standards governing the development of a smart grid. However, FERC noted that the electric industry started moving ahead with smart grid technologies prior to these government initiatives. The Commission is proposing to establish some general principles that the smart grid standards should follow.

We have known for some years that the trend was toward more numerous smaller power plants. The handwriting has been on the wall since the introduction of a feed-in tariff (FIT) system in 1978. A feed-in tariff is an incentive structure to encourage the adoption of renewable energy through government legislation. The regional or national electricity utilities are obligated to buy renewable electricity (electricity generated from renewable sources, such as solar photovoltaics, wind power, biomass, hydropower and geothermal power) at above-market rates set by the government. The higher price helps overcome the cost disadvantages of renewable energy sources. The rate may differ among various forms of power generation.

FIT means that any Tom, Dick, and Harriett with access to enough cash can set up a generating station, then sell the power to utilities, which are obliged to buy it. This model works well for facilities, such as hospitals and certain manufacturing operations, that need to maintain back-up power generation plants in the event of power failure. Most of the time these generators stand idle. FIT allows their owners to defray some of their cost by running them during peak periods, when demand may exceed fixed-power plant capacity and electricity costs (and FIT repayments) are largest.

The unintended consequence, of course, was a more chaotic electricity environment. Specifically, since a hallmark of chaotic systems is scale invariance, departures from nominal expanded to higher spectral frequencies with smaller amplitude signals (amplitude varies inversely with frequency. While these departures are smaller, their higher frequency translates into the need for faster response. Utilities began experimenting with smart-grid technology in hope of reigning in chaos over a much larger bandwidth.

ADDITIONAL RESOURCES:

U.S. Department of Energy Smart Grid

IBM Smart Grid

American Superconductor Smart Grid: It's More than you Think

Technical market analysis applies mathematical analysis of patterns in 2-D data sets to stock market (actually, any financial market) buy, sell, and hold decisions. The two dimensions are, of course, market price and time. This entry looks at one reason such data sets should be viewed as chaotic systems.

You've all heard that the so-called Butterfly Effect is a characteristic of chaotic systems. It is the fact that small, seeming unrelated happenings can have major effects on results in a chaotic system. The classic version says that a butterfly flapping its wings in China (or some other remote location) can cause a hurricane to hit South Florida.

Since few of us have a real feel for how global weather patterns develop, we all tend to say: "Yeah, yeah. Blah, blah, blah. I've heard it before, and sure it might be real to weather patterns, but it's not real to me."

This story, however, illustrates in a way we all can relate to how the Butterfly Effect works in another chaotic system -- financial markets of all kinds. I offer it up to help unbelievers understand that these markets are truly chaotic. How chaos rules stock market price movements, and how to understand its effects, is mathematically complex, but the starting point is to accept that accurate financial market analysis is impossible without using chaos theory.

That's not to say technical market analysis based on chaos theory tells the whole story. The full development shows that fundamental stock analysis is necessary to understand the forces driving stock-price movements, which technical analysis only describes.

This story depicts characters and events that are fictitious. Any similarities.... You know the rest. That the story depicts how real people and events interrelate you can judge for yourself.

In 1992, Xin Hua was a wildlife photographer in China on assignment as part of a Peoples Republic grant to Professor Yau Khan of Beijing University to study insect flight dynamics for the purpose of improving the performance of high-speed aircraft. Prof. Yau wanted to obtain high-speed video of butterflies in free flight, rather than tethered in a wind tunnel.

Such free-flight photography is difficult because, unlike the situation in a wind tunnel where the insect is constrained to be stationary in the vision system's field of view, the insect is free to move in three dimensions, while the camera must follow it. For this reason, Prof. Yao hired Xin, reputed to be the best photographer in China for this type of assignment, rather than try to do it himself or assign it to a graduate student.

Xin spent weeks trying to obtain a few minutes of film that would meet Prof. Yau's specifications. The weeks turned into months as Xin filled dozens of CDs with hours of clips showing hundreds of different butterflies flitting across fields in rural China. Finally, he had an hour or so of video that could be used in Prof. Yau's study, along with days worth of video that was beautiful, but not quite what Yau needed.

Being one of China's hungry young entrepreneurs, Xin negotiated a contract with Beijing University and the People's Republic Central Committee that would allow him to market his excess butterfly video worldwide through stock photography service Corbis. Xin shared the royalties with Beijing University.  The Central Committee was pleased to have economic stimulus.

Akira Matsumori was an advertising executive at a firm in Tokyo, Japan. One of his firm's client's was the Japanese Tourism Bureau, and, in early 1994, he was assigned to create a wonderful new ad campaign to promote the Cherry Blossom festival to potential tourists throughout the world. Matsumori developed a campaign composed of a number of ads that all climaxed with a butterfly landing on a cherry blossom.

There was no way he was ever going to obtain actual footage of a butterfly flying in from the left to land on a cherry blossom located on the right side of the screen, with all of the action limited to the bottom third of the frame, so he hired the American film graphics company Industrial Light and Magic to create a lifelike sequence using CGI technology.

To ensure that the butterfly flight was realistic, ILM artists purchased stock video from Corbis showing slow-motion butterfly free flight. The video they studied was, of course, Xin Hua's, because it was practically the only slow-motion close-up video of butterflies in free flight existent.

From Xin's video, the ILM artists developed models of how a butterfly's wings would move if it were flying horizontally from the left and landing on a cherry blossom on the right. They then used thier models to render a butterfly animation to match Matsumori's art director's specifications. They gave the butterfly a dramatic wing coloration pattern based on stock photos of North American Monarch butterflies, with black borders and an arresting gold-color fill.

By the Fall of 1994, the Matsumori's ad campaign was ready to roll out worldwide to generate interest for the 1995 Cherry Blossom Festival in late Spring. By January of 1995, the ads were blanketing television channels all over North America, from Mexico to Canada.

Maria Delgado was a travel agent in Tijuana, Mexico. Her husband, Manuel, had a highly successful leather-goods store catering to tourists. Part of Manuel's sales strategy was to mark relatively high prices on his goods, so that he could give deep "discounts" during protracted negotiations with customers. This was lots of fun for the tourists, and gave them a sense of accomplishment by negotiating the deep discounts. Manuel's shop did very well.

Manuel brought in a large stock of handbags, boots, leather jackets, and pants to sell during the annual motorcycle run starting from downtown San Diego, California and ending in a big parade along Tijuana's main thoroughfare. It was to be a huge tourist marketing event, and Manuel's shop was busting at the seams!

The week before the motorcycle run, Maria saw Matsumori's Cherry Blossom Festival advertisement. Being in the travel business, she had arranged vacations for hundreds of tourists, but, being an aggressive Mexican entrepreneur like her husband, she had never taken a vacation herself. Matsumori's ad made the travel bug bite her hard!

She loved the exotic scenes of Japanese landscapes. She wanted to go shopping in the Ginza. She wanted to visit Buddhist temples. Especially, she absolutely fell head over heels in love with ILM's CGI animated butterfly. It was so lifelike and moved so beautifully.

Making an instant decision, she threw caution to the winds and negotiated a practically free flight for her, Manuel, and the children to go for a month-long tour of Japan during Cherry Blossom time. With her travel-business connections, she obtained cut-rate accommodations near all the great Japanese tourist attractions. It was going to be a huge surprise for Manuel when she told him at dinner that night.

Oh, it was a surprise alright. In fact, it was more of a shock. "We can't do that! My shop is full to bursting with stuff for the motorcycle run as well as the stuff for the rest of the Summer. I can't just walk away and leave it for a month."

"We can afford it," countered Maria, "We've been doing very well. We can afford to take a month off for the children to see Japan before they grow up. Besides, I can't cancel all these reservations. I had to put down deposits and everything."

"But, all our cash is tied up with this stock."

"You'll just have to sell it during the motorcycle run. Clear the store out , and we'll have cash to enjoy our trip."

During the mid-1990s, my wife, Bonnie, and I were living in western Arizona, and we attended the Tijuana Run every year with friends from the local motorcycle repair shop across the border in Needles, California. In 1995, I'd just sold a struggling magazine I'd started and been trying to keep afloat. For the first time in a couple of years, we had a little money in our pockets.

After the motorcycle run and the parade, we settled down for some serious shopping. Bonnie was still carrying around the beige handbag I'd bought her several years before. It certainly didn't go with her black leather motorcycling outfit, and was well past its prime, anyway. So, she was in the market for a replacement that would be more appropriate. She found it in Manuel's shop.

Now, I never really enjoyed the kind of aggressive negotiations Manuel specialized in. I don't much like confrontation, and don't much care whether a pocketbook costs $30 or $50 as long as I have the cash and think it's not a ripoff. Bonnie, however, likes negotiating a bargain. So, I always let her do the deals.

"You pick out anything you want, and get the price you want. I'll pay for it," is what I told her.

Because Manuel's travel-agent wife had fallen in love with an animated butterfly based on a real butterfly photographed by a Chinese wildlife photographer, the market price for that handbag in that shop at that time on that morning was $15 -- well below the $45 I'd have expected to pay for it in the States.

Notice that the handbag did have an intrinsic value based on the cost of the materials, labor, energy to make it and transport it to Manuel's shop, along with carrying costs Manuel incurred by keeping it in the store, and having a store to keep it in. That intrinsic value is the value a financial accountant would arrive at by fundamental analysis.

It is not, however, the market value. The market value is simply set by two parties negotiating one sale. It's based on a host of factors including, but not limited to, the two parties' ability to negotiate, their desire to make the sale, and the amount of coffee they'd drunk within one hour of starting the negotiations.The intrinsic value of the the item is only one factor among many.

In this case, the flapping of the Chinese butterfly's wings had a profound affect on the two parties relative willingness to make a deal, which had a major affect on the price arrived at.

The take home lesson is that hundreds of such seemingly unrelated events affect each and every sale of a financial instrument, whether the deal is struck between amateurs, professionals or 10 year olds swapping baseball cards. That fact makes stock markets -- indeed all markets -- chaotic systems.