Equipment Reliability Institute
ERI News - your reliability newsletter
August 2003 - volume 12

Hello, everyone!

This issue starts off with some ideas by Rob Poltz concerning saving money, a subject which should interest everyone. By expending effort "up front" you save many times more money in future years. Rob will expound on his ideas September 15-17 here at Santa Barbara.

Are you feeling hot, humid and dusty this August? You're uncomfortable, but probably not in danger; Dave Douthit details the dangers being experienced by your newest printed circuit cards.

Bob Renz of General Dynamics - Advanced Information Systems at Bloomington, Minnesota, offers some practical suggestions to those who operate electrodynamic shakers. He also comments on Wayne's answer (last issue) to a reader-asked question on accelerometers.

Please, readers, ask questions. You can e-mail us or you can also visit our message boards at ERI website or at vibrationandshock.com.

Best wishes,
Wayne Tustin

What is the Cost of Unreliability?
by Robert Poltz

As consumers, we purchase products, expecting that they will work correctly the first time we try them and every time thereafter. But we are sometimes disappointed. Fortunately, however, the product usually comes with a warranty and we return it to the place of purchase for a refund or credit. This scenario works nine times out of ten, on average. If not, we usually chalk the purchase up to experience and never go back to that manufacturer again.

But, what if you are that manufacturer, trying to sell good, reliable products to the public? Companies that pay attention to Reliability and Quality succeed. Those that don't, fail. It is as simple as that.

In electronics, Hewlett Packard, Sony and Pioneer come to mind. Each name engenders immediate association with quality and reliability of their products. Consumers return to the brand name for subsequent purchases because they trust these companies to produce the best bang for their buck.

In automobiles, we have BMW and Daimler-Chrysler (Mercedes-Benz) identified as tops in their respective fields because of high reliability and quality standards of excellence.

These are brands I trust and have used for twenty years. In all likelihood, I'll purchase another of their products in the near future. Your experience with other manufacturers may differ from mine, but my point is that consumer loyalty is based upon trust in products. Reliability of design is essential to creating such consumer confidence. Quality is the implementation of that design in manufacturing and delivery to the consumer.

Barringer [1] writes that "the cost of improvement efforts are more productive when motivated from the top-down rather than the bottom-up, because it is a top management-driven effort for improving costs." Management steers the company, and top management are at the helm. Decisions flow downward to supporting organizations within the corporate structure to implement these ideas.

Cost of unreliability programs are the spearhead for reliability improvement. Appropriate questions are:

• Where within the organizational structure is the cost problem(s) located?
• What magnitude is the problem(s), are all business units affected, how can the problem areas be contained?
• What are the principal causes of these problems, funding cutbacks, labor shortages, scheduling, transportation?

A useful management tool for predicting product success/failure) is the Life Cycle Costing Model, LCC. Each of the aforementioned areas is an integral part of the various models. Dhillon and Reiche [2] co-authored a book entitled Reliability and Maintainability Management and in chapter 13, "Life Cycle Costing and Warranties" they discuss the need and uses of LCC.

Life Cycle Cost enables management to:

• choose between competing manufacturers, products and services
• compare costs of alternative approaches to meet a specific need
• make equipment replacement decisions
• stimulate orderly planning and scheduling
• gain control over ongoing program costs

The general mathematical model for LCC is:

LCC = kD + kRD + kOS + kP

KD is the cost of disposal
KRD is the cost of research and development
KOS is the cost of operation and support
KP is the cost of production

Within the context of Reliability, all these are of great interest throughout the life of the product or system. Each element plays a significant part in the design for reliability equations from Infant Mortality, Useful Life and Wearout of the product.

Systems Effectiveness is an index of value used to evaluate product/system designs as a measure of desirability to create satisfaction divided by the price of the item (Ireson 1996). Systems Effectiveness uses the LCC as the quantity of price and uses effectiveness as the quantity measure of results received.

The "Effectiveness equation, therefore, is the product of the chance that equipment or systems will be available to perform its duty, it will operate for a given time without failure, it is repaired without excessive loss maintenance time and it can perform its intended production activity according to the standard."[3]

As related to engineering, each element of the effectiveness equation is premised on data that varies by product and specifies a probability of success between 0 and 1.

Within the RAMS disciplines there are Availability, which deals with whether an item is up and running and available for service. Reliability, which deals with whether an item fails over a time interval and predicts the odds of failure vs success. And then there is Maintainability, which concerns itself with the duration of maintenance activity that will restore an item to service. Often overlooked is Capability, which deals with productive output of a product or system, or how well it performs compared to some specification.

In summary, one must look at all aspects of all factors contributing to the success/failure of a product. Try to develop a global perspective of events within your organization, identify potential weaknesses in structure and plan for success by using forecasting models. Invite a Reliability expert to provide quantification data for these models through detailed systems analyses.

These topics are the subjects of my upcoming seminar/workshop, September 15-17, 2003, in Santa Barbara. For more information or registration, click here.

Bibliography
[1] Barringer, "Cost of Unreliability" http://www.barringer1.com/cour.htm
[2] Dhillon and Reiche, "Reliability and Maintainability Management", Van Nostrand Reinhold, 1985.
[3] Barringer, "Life Cycle Costs & Reliability for Process Equipment", 1996.

Robert Poltz is Reliability Consultant, President and CEO of Design Analytx International, worldwide leader in reliability of design analyses of high tech systems. For more information about Robert or to contact him, click here.

(back to the top)

The True Cost of COTS

The sure sign of momentum is when a movement acquires its own acronym. The current enthusiasm for commercial off the shelf technology is known as COTS, and one avionics expert suggests it may be prudent to rein in the present enthusiasm.

In the rush from shelf to application, David Douthit cautions, "It is impossible to produce a high-reliability product without accurate and thorough environmental testing."

In this respect Douthit, owner/manager of a Mesa, Arizona-based avionics reliability consulting firm, LoCan, LLC, seems to be recalling the dictum of 19th century British prime minister Benjamin Disraeli to "make haste slowly." Or, rather, temper the enthusiasm with testing.

It may be useful to highlight certain trends, by way of establishing the basis for Douthit's concerns. For one thing, the military market for electronics is shrinking and military specifications for reliability of components no longer apply. An entire infrastructure that once guided component reliability for both military and commercial application has dried up like a hail pellet on a slab of sun-baked asphalt.

Ever thinner, ever more vulnerable
For another, airplanes and other military vehicles are being outfitted with electronics and wiring that may degrade during service. Printed circuit boards come to mind as one primary example. To be sure, they pack performance into a small space, but how is it done? The circuits are layered on top of one another, 25 or more in a board just slightly thicker than a tenth of an inch. As Douthit pointed out, "This overall reduction in spacing and thickness, combined with the increased density of the circuits, increases the sensitivity of printed circuit assemblies to moisture and ionic contamination." Ionic contamination is the process of metal erosion when electrically charged microscopic particles are deposited on the metal's surface.

Integrated circuits, once packaged in ceramic, are now more often encased in polymer resins, giving rise to another acronym, PEMs, for plastic encapsulated microcircuits. Unlike rock-hard ceramic, these materials are vulnerable to penetration by contaminants and, worse, by moisture.

Coatings, designed in part to prevent penetration, introduce another variable. More than 400 different materials are now being used as coatings. There is a huge variation in their coefficients of thermal expansion. If not accounted for, electrical components can become detached or damaged, especially if coatings with higher thermal expansion characteristics are used in areas of the vehicle likely to experience significant shifts in temperature.

The dust devil
When packing density goes up, so does the circuitry's potential to generate heat. Designers have resorted to forced-air cooling to carry away the heat. As in all things, there are side effects. Douthit maintains that too little attention has been devoted to airflow patters inside the boxes housing these components. Venturi effects, he maintains, can cause very high rates of airflow. More is not necessarily better. The higher airflow can increase up to a hundredfold the rate at which dust is deposited on delicate components. Not only is the dust a fire hazard, it can conduct electricity, leading to intermittent short circuits and computer failures.

Industry experts now recognize a finer grain of dust, made possible by improvements in measuring technology. Once measured in the range of .01 to 2 microns, a new "ultra fine" class involves particles ranging from .002 to .01 microns - extending the scale of smallness by a factor of five.

Thin circuits under thin coatings, blanketed by thin layers of contaminating dust - these are just three factors affecting reliability of printed circuit boards.

Wiring has its own trilogy of threats. Douthit points out that thinner insulation has been used to save weight on wiring. Again, for every benefit there is a countervailing problem. Aromatic polyimide insulation is weakened by water. The water removed during the manufacturing process makes this type of insulation a sponge when exposed to high humidity and water condensation in service. With thinner insulation of any type, more care must be paid to preventing dents, crimps, nicks, gouges, sharp bends and pressure points, which can break already thin insulation.

Crimp-type wire connectors pose another problem. Although protective coatings are often used, crimped connections can become casualties of corrosion. Even "gold to gold" contacts are not immune, Douthit claims. "Gold plated contacts usually have a layer of nickel under the gold. If the gold is worn and the nickel is exposed, oxidation will cause serious intermittent problems," Douthit maintains. Single sided crimp connections are best soldered, whenever practical, to provide the strongest barrier against breach, he suggests.

As it all of this weren't enough, these problems are exacerbated in harsh environments. Indeed, SWAMP is the telling acronym for severe weather and moisture prone areas of the aircraft deemed the greatest threat to the safe functioning of electronics - the leading and trailing edges of the wings, landing gear wells, and the juncture of wings to fuselage.

Corrosive cocktail
These areas mark the front lines of exposure. Remove inert nitrogen, which constitutes 80 percent of the atmosphere, and what remains is a corrosive cocktail of oxygen, moisture and contaminants. At atmospheric pressure, a downpour of this cocktail's molecules (400 x 1014 to be more exact, or 400 followed by 14 zeroes) will strike a square centimeter (0.4 square inches) every second, Douthit calculates. In a typical urban environment, the constituent compounds include chlorine, iodine, trace metals, sulfuric acid, nitric acid, hydrochloric acid, sulfates, nitrates and, in coastal areas, airborne sea salt. Up to 50 percent of the summertime haze, often found in the skies over the northeastern United States, is a combination of sulfuric and nitric acids.

Moisture is a superb solvent and catalyst, abetting the penetration of corrosive compounds through a coating as far as the metal surface. The higher the humidity, the worse. At 60 percent relative humidity, the moisture forming on the surface of an electrical component may be only 2-4 molecules thick, but at 80 percent humidity, the layer may be 5-20 molecules thick. At or above a thickness of five molecules, the moisture layer is the same chemically as bulk water. In some situations, a thicker layer can facilitate more energetic chemical reactions. "Microscopic points of attack will have already been started because many of the particles carry water with them when they are deposited," Douthit observed.

These dour details all feed into the COTS conundrum. "The primary issue is what is a harsh environment?" Douthit said. Traditionally, three minimum conditions have applied:

1. A relative humidity of 60 percent or higher,
2. A temperature above 0º C (32º F), and,
3. A deposit rate of contaminants on the circuitry surface exceeding 1.5 micrograms of salt, or its chemical equivalent, per square centimeter during the expected lifetime of the product.

Of these criteria the last is the least certain, because circuits of newer designs may be more subject to contamination. Some new densely packed circuits feature a salt contamination limit of 0.2 micrograms or less, roughly a sevenfold reduction in the traditional limit. To Douthit, the wider application of inadequately tested COTS technology represents a journey into unknown territory. "The result is an increasing amount of intermittent failures, which cannot be verified or confirmed as equipment ages."

The COTS conundrum
Result? The illusory cost savings associated with COTS technology tends to be offset over time by the increased cost of trouble shooting and maintenance.

Indeed, support for Douthit's concern comes from two sources. Earlier this year, operators were warned that epoxy molding compound containing phosphorous has led to instances of internal electrical shorting. The May 2002 ADVISORY, published by the Computer Aided Life Cycle Engineering (CALCE) office at the University of Maryland, cautioned that phosphorous' affinity for water led to "phosphorous particles bridging wire bonds," making it a "failure accelerator."

It seemed that the standard mesh size worked for bromide, but not for phosphorous when the manufacturer switched to its use as a substitute fire retardant. Since then, a finer mesh has been used to sieve the material, but bromide-braced components out in the system have demonstrated failures within 6 to 12 months of operation.

This case illustrates that process control tests used in production are not the same thing as reliability tests.

The second source of support for concern about COTS comes from the Federal Aviation Administration (FAA). Its August 2001 review of COTS electronics in airborne systems, presented three important points. The first is that building high reliability systems using COTS can be more expensive than utilizing hardened hardware built to military specifications. Second, the introduction of new materials poses new reliability risks. The compatibility matrix is approaching infinity.

Third, reliability research is much more fragmented in the commercial sector than in the military community, and much of the limited information that does exist is considered proprietary.

What is to be done? Douthit has a couple of suggestions. More research on COTS component reliability under field conditions is needed, he maintained. Of greater importance, he says, "A new paradigm will be required."

"A central agency, such as the FAA, must step up to provide the industry with a centralized source of information on COTS components." Rubber-stamping of military standards "tweaked" for commercial application represents not only an "enormous conflict of interest" but also presents a "major potential for disaster," Douthit asserted.

"Until proper test equipment and protocols are developed, there is a huge 'black hole' in avionics reliability," he maintains.

A sobering statement in the FAA's report of the enthusiasm for COTS buttresses Douthit's concern: "Commercial market trends are rapidly diverging from the needs of safety critical airborne systems."

The lesson du jour: when a cost saving looks too good to be true, it probably is.

Test Lab Musings (part 1)
by Robert L. Renz

Vibration and shock levels always seem to be increasing, so your shaker will always be on the edge of its capacity. Be careful when designing a fixture to watch its weight - paying the extra for magnesium up front might make the difference between being able to run a profile, or having to go to an outside lab.

There is nothing as eternal as a test lab. We never really abandon anything, we just move it to a different shelf. The old tape cassette deck that we used to drive the shaker is kept on a shelf just in case we ever need it again... The manual equalization panel that goes back forever? On a shelf, right behind the antique spectrum analyzer and right next to the boxes of new 5 1/4 floppy disks. Naturally, the cassette tapes, the connecting cables, and the instruction manuals were thrown out years ago. The new equipment goes on the most accessible (and visible) shelves while the older equipment gradually crawls deeper and deeper into the depths and shadows of the lab.

And why is it that no one has ever standardized on the size for hardened washers for the fixture bolts? Over the years, I've run into 3/8" washers with OD's from 1/2" to 7/8", and thicknesses from 1/16" - 1/4". As challenging as this makes it when you are trying to bolt down a fixture, it's even worse when you need to use a fixture that turns out to be counterbored for a 1/2" OD washer - and you can't find enough of them..... At that point, I run the fixture to the machine shop and have them re-counterbored to 7/8". Instead of having the machine shop make washers from solid bar stock, I buy hardened washers off-the-shelf from a machine tool supplies dealer at a cost of only about 50 cents each. Likewise for bolts - Buy 'em by the box, and don't be afraid to throw them away when they start to get beat up. And while we're at it, standardize on one diameter bolt whenever possible, such as 3/8". And also, limit yourself to either fine or coarse threads- one or the other. Wait until you have both fine and coarse threads in your lab, and you're trying to set up for a test....

Many labs use a battery-operated drill to run down / spin out fixture attachment bolts, usually with a kluged adapter to adapt it all to a hex wrench with a long enough drive to reach down into the fixtures. Instead of using this pile of adapters, extensions, and sockets held together with duct tape, it's a lot easier to use a 12" (or longer) hex wrench chucked directly up in the drill. Can't find long wrenches? Check the Bondhus catalog or on line at www.bondhus.com - they are in there (1/4 x 12" Balldriver Blades are part number 3612, 5/16 x 12" Balldriver Blades are part number 3613, 1/4 x 36" hex stock is part number 191912, and 5/16 x 36" hex stock is part number 191913). Your regular Bondhus distributor can order them for you.

Rober L. Renz of General Dynamics - Advanced Information Systems at Bloomington, Minnesota.

(back to the top)

Questions our readers have asked...

This section of our newsletter was created for you, reader! Feel free to send questions or suggestions to the webmaster. One of our specialists will respond.

In our last issue, Wayne responded to the question "How many accelerometers do I need?". Robert L. Renz wrote back and would like to share his comments with us:

"Hello, Wayne.

As you said, the total can reach 12 very quickly.

In addition, though, we also have to consider what type of test it is.
Are we evaluating a satellite module that will be very hard to get to
after it is launched, or are we looking at an engineering prototype that
will be back and forth to the vibe lab for many months? Is it in our own
lab and under our control, or is it at an outside lab? Is the UUT a
complicated unit, or is it a monolithic shape?

In some cases, our customer tells us what they want to measure, which
simplifies the whole matter a lot. I always make sure that I have extra
accelerometers & cables available in case one dies in mid test,
especially if the customer wants to witness the test.

If, however, you are instrumenting a MIL-STD-901 heavy shock test for
COTS equipment, you will discover that you need N+1 accelerometers,
where N is the number you have installed... The problem is that every
accelerometer channel usually costs about $ 1000 per shot, so you are
caught in the middle between engineering and the money people. I usually
plan on about 12 accelerometers per shot, and try to get some of them
positioned near the COTS equipment within the cabinet. Otherwise, when
the COTS equipment goes obsolete and has to be replaced by something
else, you will have some idea what shock / vibe levels that the
replacement will have to handle. Remember, in a barge shot you will have
the shock wave hitting the barge at angles approaching 45 degrees
vertically, so you will have to monitor side-side motion as well as
up-down motion, which eats up a few more accelerometers."

Rober L. Renz of General Dynamics - Advanced Information Systems at Bloomington, Minnesota.

(back to the top)

My new text is titled "Random Vibration and Shock Testing, HALT, ESS and HASS". We're describing it as "A minimal-mathematics introduction to the fundamentals of Random Vibration and Shock Testing, HALT, ESS and HASS also Measurements, Analysis and Calibration, with applications in the fields of aeronautical, automotive, seismic and shipboard design and production". It is shown here, alongside my 1984 text "Random Vibration in Perspective", which is updated and greatly expanded.

Click on the image to enlarge

The book will be approximately 300+ pages, letter size, hard cover and in color. It will include a CD-ROM, which carries a number of video clips and animations that illustrate vibratory and shock motion. One of my favorites shows the highly repetitive motions inside an automobile engine. Another shows cantilever beams responding to random vibration.

Prepublication Sale
You can order the book for only $150 if your funds reach us by December 1, 2003. And we'll pay the shipping. After that the price will be $250.00 + shipping.. We expect to ship by February 1, 2004.

Reference edition of 1001 CD Program

Many of our readers are aware of ERI's 1001 Distance Learning Program in vibration and shock technology. The participant receives a CD-ROM that has some 3000 PowerPoint® slides in 31 lessons. At the end of each lesson, the participant keys in answers to problems and questions in the lesson "review", then attaches his work to an e-mail to his "remote professor", Wayne Tustin. Wayne personally reviews each lesson, sometimes asking the participant for additional work. Eventually the participant receives his/her Certificate of Completion. Details may be found at our website.

A few people have told us "I don't want to perform 31 lessons and I don't need your Certificate. How can I have the CD only?" Now you can purchase the "Reference Edition" CD. Click here to order one.

Vibration and Shock in Switzerland

"Vibration and shock training surrounded by the Alps" is one way to describe Markus Dumelin's event at Thun, Switzerland October 14-16. Details can be found here.

On the same dates, Wayne will offer similar training at Newport, Rhode Island. Details can be found on our websites.

AST Meeting
Where will you be the week of October 20-24? If you're interested in Accelerated Stress Testing, click here for details of this Seattle meeting.

Shock and Vibration Symposium
On October 27-31 the 74th annual Shock and Vibration Symposium will meet in San Diego. Click here for details. Wayne is to tutor on fixture design, fabrication and usage.

Reliability Symposium
The CRMS Reliability Symposium will meet October 2003, in Ottawa, Canada. Click here to visit their website and get more information.

Acoustics Consultant Wanted
We'd like to hear from an acoustics consultant who is willing to travel and to teach short courses to groups of engineers and consultants, possibly in non-industrialized countries.

ERI - Equipment Reliability Institute
1520 Santa Rosa Ave.
Santa Barbara - CA - 93109
Tel: (805) 564-1260
Our fax number:
(805) 966-7875

Wayne Tustin tustin@equipment-
reliability.com

Webmaster webmaster@equipment
- reliability.com

Websites
http://www.equipment-
reliability.com

http://vibrationand
shock.com

Copyright © 2000-2003 Equipment Reliability Institute. All rights reserved.

Subscribe
If you would like to subscribe to ERI News, go to either website, fill in the form "Free Newsletter" and hit the Submit button. Subscribe now!

Recommend
If you enjoy reading ERI News and want to recommend it to a friend, just hit "forward" on the menu of your e-mail program or tell your friend to subscribe to it at our website.

Previous issues
Missed the previous issues? It is not a problem. Just visit our newsletter archives section and find all ERI's News issues.

Unsubscribe
If you do not want to receive ERI's quarterly newsletter, please send us an e-mail using the same e-mail address that brought you the newsletter, with "remove" as subject.