A quality management case studydefects in spacecraft electronics componentsShareShareCASE STUDY Quality Management, Complexity June $1999$Project Management JournalKuprenas, John A$.|$ Kendall, Randolph L$.|$ Madjidi, FarzinHow to cite this article:Kuprenas, J$.$ A$.,$ Kendall, R$.$ L$.,$ & Madjidi, F$.(1999).$ A quality management case study: defects in spacecraft electronicscomponents. Project Management Journal, $30(2),1421.$Reprints and PermissionsJohn A$.$ Kuprenas, University of Southern California, Department of Civil Engineering, $122$ S$.$ Larchmont Boulevard, LosAngeles, California $90004$ USARandolph L$.$ Kendall, The Aerospace Corporation, $2350$ E$.$ El Segundo Blvd$.,$ El Segundo, California $90245$ USAFarzin Madjidi, Pepperdine University, Graduate School of Education and Psychology, Malibu, California $90263$ USA AbstractProduction of spacecraft components presents a management challenge in that the systems are complex, costly to buildand launch, and once a spacecraft is launched, it is extremely difficult and expensive to repair. For these reasons, strictstandards for quality and reliability are essential. This paper presents a project quality management case study for theproduction of spacecraft electronics components as part of an overall spacecraft project. Printed circuit board data isexamined by project managers using a Pareto analysis of different defect types to reveal process problems. Correctiveactions to the production process that were implemented by project managers are explained, and ideas to improve futurequality management studies, regardless of the industry, are discussed.Keywords: quality management; quality planning; quality assurance; quality control; Pareto analysisSuccessful project management entails balancing competing demands among project scope, time, cost, and quality.Certain types of projects, however, may dictate that a manager place more emphasis on one particular element. Projectmanagement within the spacecraft industry requires a strong emphasis on quality because spacecraft systems arecomplex, costly to build and launch, and once a spacecraft is launched, it is extremely difficult and expensive to repair.Given this unforgiving nature of the spacecraft industry, production processes have to utilize strict standards for qualityand reliability. This paper describes how project managers performed project quality management of a spacecraftcomponent production process as part of an overall spacecraft project. This paper presents the data collection andanalysis, Pareto diagram creation and interpretation, and subsequent management conclusions and actions as part of aquality management study of the production of spacecraft electronics components.One of the principal PMBOK Guide processes is quality management. Quality management of any process consists ofthree tasks: quality planning, quality assurance, and quality control. A Guide to the Project Management Body ofKnowledge $($PMBOK Guide$)$ further describes each of these components. Quality planning is the identification of qualitystandards and the development of methods to satisfy them. Quality assurance is the collection of the overall actions usedto ensure confidence that standards are met. Quality control is the evaluation of results relative to standards and theelimination of causes of unsatisfactory performance $($Project Management Institute Standards Committee, $1996).$ Thestudy described in this paper illustrates how successful project quality management requires the use of all three elements.The Pareto Principle is based upon the observation of Vilfredo Pareto in nineteenth century Italy that $20\%$ of thepopulation controlled about $80\%$ of the wealth. Quality researchers applied Paretos concept to the causes of qualityfailures and suggested that, in most cases, quality losses are distributed in such a way that a vital few quality defects orproblems make up the largest portion of overall quality losses $($Juran & Gryna, $1988).$ The Pareto diagram is a graphicrepresentation of this concept. The Pareto diagram itself is a histogram with the category or cause of failure data arrangedin order from the largest to the smallest. Used in quality applications, Pareto diagrams graphically allow the separation ofthe vital few items from which the majority of defects are generated from the trivial many from which relatively few defectsare generated. Quality improvement resources are then directed to the correction of the vital few defects, thus maximizingthe effective use of available resources. Used in such a fashion, Pareto diagrams are an excellent project qualitymanagement control tool. Pareto diagrams have been used in a variety of processes requiring quality control, such aspatient waiting times $($Buckle & Stuart, $1996),$ worker safety $($Kuprenas$,$ Kenney, & Nasr, $1999),$ and engineeringmanagement $($Graves$,1993).$Problem Description and AnalysisThe project quality management study examines for defects found during final inspection of printed circuit boards beforethey are integrated into the next higher assembly, which is typically a discrete electronic component or box such as apower supply, telemetry transmitter, or flight computer. The data for the study was gathered from all the printed circuitboards manufactured by a large United States government contractor during one year for a specific military spacecraftproject. About the AuthorsJohn A$.$ Kuprenas is a lecturer in civil engineering at the University of Southern California. He is also a professionalengineer and a project manager for a construction management consulting firm and has over $12$ years experiencemanaging a variety of programs and projects. His research concentration is on computer applications in construction andon construction processes, productivity, and quality.Randolph L$.$ Kendall is a mechanical engineer and senior project engineer for the Evolved Expendable Launch VehiclesProgram at The Aerospace Corporation. He has managed several large production and design projects in the aerospacefield. He recently has completed his M$.$B$.$A$.,$ and his research interests are in planning, production, and quality.Farzin Madjidi is an associate professor of research methods, Graduate School of Education and Psychology atPepperdine University. He has over $15$ years of professional and consulting experience with a number of private andpublic organizations in the areas of management development, project management, and quality. He holds an M$.$S$.$ instructural engineering and business administration, and a Ph$.$D$.$ in institutional management.While the printed circuit boards can vary significantly in size, complexity, and end use, they are very similar in constructionmaterials and processes and can therefore be considered together as a generic class. The printed circuit board data isexamined using a Pareto analysis of different defect types to reveal process problems. The corrective action, which wastaken by the project manager, is described, along with suggestions for future process improvements.Data Collection. Since this quality management study examines a military spacecraft application, the manufacturingprocess is already tightly controlled, as defined by various standards such as MIL$-$Q$-9858$A Quality ProgramRequirements and MIL$-$STD$-1547$B Electronic Parts, Materials, and Processes for Space and Launch Vehicles. Inaddition, these standards dictate a number of requirements for the inspection, documentation, and feedback of data onthe manufacturing process. These requirements are used to ensure the detection and correction of both manufacturingdefects and process problems at the earliest possible stage. A cornerstone of this quality system is $100\%$ inspection of allelectronics boards. These preproduction efforts constitute the first of the three elements within quality managementquality planning. Outputs are a clear plan, operational definitions, and a list of items to be checked. These outcomes of aproject quality planning effort are typical of project management of any project.The basic MIL$-$Q standards are written by the government and required contractually for work on the government contract.The spacecraft manufacturer implements an internal quality system and processes that satisfy the MIL$-$Q standards. Thisquality system may be used for both commercial and government programs. Project managers supervise the datacollection effort as the project data is gathered by the inspectors from the quality assurance organization. The projectmanagers work together with the quality assurance organization to analyze the data and identify those problems that needfurther investigation. Modifications to the data collection procedures may be required to isolate the root cause of aproblem. This is part of a continuous improvement process for the project wherein the manufacturer is continuallyexamining its processes and trying to correct the most significant causes of defects.It should also be noted that the United States government, in the last couple years, as part of the acquisition reforminitiative, has been moving away from the use of MIL$-$STDs and toward the use of commercial standards and IS$0-9000-$based quality systems.Figure $1.$ Number of Defects Found per WeekDiscrepancy Reports. When a defect is detected during an inspection, a discrepancy report $($DR$)$ is generated. A DR is adata record in a database that contains information on where, when, and by whom the defect was detected, the partnumber of the printed circuit board involved, and the type of defect detected. The DR also documents the action taken torepair the defect and, if appropriate, the process corrective action implemented to avoid similar problems in the future. AllDRs are collected into an online database for use by the project manager in the project quality management process.The DR database remains intact for the life of the spacecraft program and can be used to track specific or genericproblems. The database can be searched for various parameters such as DR number, part number, or defect type. Alldefects are classified by type and given a three$-$letter defect code in the database. Within the project of this case studythere are $31$ broad classes of defects. Three of the simplest defect classes are wiring, soldering, and welding.These broad classes are then further broken down into over $200$ specific defects. For example, the general class ofsoldering defects includes $23$ specific defects such as missing soldering, insufficient solder, poor wetting, and incorrecttinning.These planned inspection and data$-$gathering efforts constitute the second component of quality managementqualityassurance. The output of this quality assurance process $($creation of the DR based upon the MIL$-$Q standards$)$ is qualityimprovement with respect to effectiveness and efficiency of the project. Again, these outcomes of a project qualityassurance effort are typical of project management of any project.Data Analysis. The DR system was used to gather data for a one$-$year period on all of the defects found duringinspections of printed circuit boards built for a specific spacecraft program. Data analysis by project managers began withan examination of the number of defects found and the number of inspections performed each week. Figure $1$ shows thetotal number of defects found per week for each of the $52$ weeks during this case study. As shown in Figure $1,$ there iswide variation in number of defects discovered per week. An initial quality control interpretation considered by projectmanagement was that major quality problems may have existed in weeks $5$ and $10($since for each of these two weeksover $500$ defects were discovered$)$ and that, for some reason, quality improvement was made toward the end of the year$($since the number of defects found per week for any week past week $35$ never exceeded $250).$Figure $2.$ Number of Inspections per WeekHowever, this data alone may be misleading, and lead to improper interpretation and management actions. Accordingly,additional data was examined. Figure $2$ shows the number of inspections in each week. There is large variation in thenumber of inspections per week and, therefore, a correspondingly large variation in the number of opportunities fordetection of defects. Project managers needed to explain both this wide variation in inspections and how it affects theinterpretation of the defect data. In order to correlate the defect and inspection data, a Pearsons coefficient of linearcorrelation was calculated. The Pearsons r was found to be only $+0\mathrm{.}11.$ This value of r is considered a low coefficient;hence, little correlation exists, between defects and inspections.The next step the project managers used in the data analysis was to calculate the ratio of defects to the number ofinspections for each week. The plot of the ratio of defects found per inspection is shown in Figure $3.$ This data makes thequality picture much clearer. Managers saw that there may have been minor problems in weeks $5$ and $10,$ but the realquality problems began somewhere between week $22$ and $26.$ Quality problems continued to worsen through week $31.$ Acorrective action is evidenced starting in week $32,$ with the process improving by week $37.$Pareto Analysis. In order to find and correct the specific problems previously noted, project managers needed additionaldetailed data on the actual types of defects occurring. In addition, a comparison of data for a week in which quality wasgood $($low number of defects per inspection$)$ with data from a week in which quality was poor $($high number of defects perinspection$)$ was desired. Hence, a Pareto analysis of defects at the general class level was conducted. This Paretoanalysis was the third component of quality managementquality control. Outputs are complete checklists, processadjustments, rework, and additional quality improvement. Again, these outcomes of a project quality assurance effort aretypical of project management of any project.In order to do this quality control task, defects were tabulated by general category for weeks $6$ and $31.$ As the ParetoPrinciple suggests, not all of the $31$ general categories of defects are represented in significant numbers. Therefore, onlythe six most common defects were further examined. Starting at week $6,$ the defects by type are shown in a pie chartformat in Figure $4.$ Figure $5$ shows the same $($week $6)$ data in a Pareto diagram. This Pareto analysis showed that evenduring week $6,$ when the production quality was high, there are still some defects, which dominate the rest. In week $6,$ themain errors were soldering problems, component defects $($electronic piece$-$parts in this context$),$ documentation errors$($i$.$e$.,$ lack of operator initials at mandatory procedure steps, etc.$),$ and exposed board weave. These four vital defect typesout of $31$ accounted for $77\%$ of all of the defects.Figure $3.$ Defects Found per Inspection by WeekA Pareto analysis of the week $31$ data showed that not only did the total number of defects and defects per inspectionincrease but also the percentage of defects caused by the key defect types increased. Figures $6$ and $7$ show the pie chartand Pareto diagram for week $31.$ The project manager noted that in week $31$ the top three defects now accounted for $80\%$of all defects and the vital top four defects accounted for $89\%($as opposed to $65\%$ and $77\%$ in week $6).$ The top fourdefects are the same soldering, documentation, components, and board weave. The documentation defects, however,are now more numerous than the component defects. Soldering defects alone now account for $41\%$ of all defects, asopposed to $28\%$ previously, and documentation defects account for $22\%,$ as opposed to $15\%.$ Based on these Paretodiagram analyses, quality improvement efforts focused on soldering and documentation problems.Management ActionsWorking group teams were formed to study the soldering and documentation processes, determine the root cause of theproblems, and suggest corrective actions. A typical working group team was composed of the spacecraft programmanagement organization project manager, representatives from the quality organization, the manufacturing organization,and usually a customer $($government$)$ representative. Both working$-$level and management personnel were represented.The team usually met at least weekly at the production facility to review the latest data and discuss possible correctiveaction. The meetings continued until the problem was resolved.As a result of the work and research of these teams, actions were taken that improved these processes. In the solderingarea, procedures and drawings were reviewed and clarified. In some cases, tooling and equipment were upgraded,operators were given additional training, and visual aids were developed to show acceptable verses unacceptablestandards. In the documentation area, procedures were upgraded and operators were given additional training.Figure $4.$ Defects by Type for Week $6$Figure $5.$ Pareto Diagram of Defects in Week $6$One of the trends observed in the data, which is not explained, is why the number of inspections went down so muchduring the problem period. There are several possible explanations. First, the number of inspections is directly related tothe rate at which work is being completed. As defects increase, the amount of rework to be done increases, and theamount of work completed decreases since overall finished product productivity is decreased by the rework. Thereworked boards, however, still need to be inspected.Another possible reason for the decrease could be increased numbers of defects because of inexperienced operators$($due to a turnover problem$).$ If this were the case, the operators would likely be completing work more slowly anyway,again manifested by lower productivity and, hence, fewer inspections. This phenomenon will be further studied in futurequality management efforts. This study provides a collateral benefit to the overall program management organization inthat future project quality management efforts can be alert to this trend and additional analysis can be completed.Figure $6.$ Defects by Type for Week $31$This analysis also highlights a problem with the data$-$gathering system. In this study, data are collected only for defectsand inspections with just one inspection per printed circuit board. The printed circuit boards, however, have a wide varietyof size and complexity. Therefore, opportunities for defects on each board can vary. For instance, a shop may be workingmainly on boards for a complex inertial measurement unit one month, while the next month they are working mostly onsimpler boards to go into a fuse box unit. The complex boards produced in the first month present many more potentialdefect opportunities than the simple boards of the second month. Thus, variations in number of inspections done andnumber of defects per inspection is probably correlated to the type of work being done during a given time period.One way to eliminate this factor in the data would be to count the number of defect opportunities $($approximately thenumber of electronic piece$-$parts$)$ per board type and normalize the defect data to defects per opportunity rather thandefects per inspection. This change in the data collection would definitely give more accurate data but it would increasethe complexity and cost of an already complicated system. The current system does seem capable of successfullyidentifying major problems, and the added utility may not be worth the cost.The project quality management effort illustrated in the case study is fairly typical of spacecraft electronics components. Astandard set of metrics is used to track total defects and primary causes. When a specific problem is identified, moredetailed information is gathered and analyzed. Future efforts usually rely on past data for trend analysis. In addition,corrective actions for a specific problem may include systemic changes in the quality processes for data collection and$/$oranalysis$.$ Once again, it should be emphasized that the quality process is one giant feedback loop with the goal ofminimizing product defects. All aspects of the system are candidates for improvement, including supplier quality, incomingparts and materials inspection, manufacturing processes, and design changes to make the product less susceptible toworkmanship problems $($design for manufacturability$).$For all types of projects, successful project quality management requires a responsive quality system to identify processproblems before they get out of control. If products are continually rejected or must be reworked, this will have a seriousnegative impact on both the cost and schedule of a project. Often, alternate suppliers or manufacturing shops $($secondsourcing$)$ must be located to ensure reliable manufacture and delivery of components on the critical path for a givenproject.ConclusionSuccessful project management must include project quality management. As part of ongoing project quality managementefforts for a large United States government contractor, a study was conducted of the manufacturing defects encounteredduring one year of production of spacecraft electronics components. Data was gathered from an extensive databaseproduced by $100\%$ inspection of all printed circuit boards manufactured by a production contractor. While this inspectionsystem is costly$,$ it is justified in the military space application because of the nature of the product produced.Figure $7.$ Pareto Diagram of Defects in Week $31$A Pareto analysis of the data was conducted and proved to be a valuable quality control tool by effectively identifying thekey manufacturing problems and suggesting areas in which to implement corrective action. Cost$-$effective correctiveactions were in fact implemented by the contractor and were successful in improving the manufacturing processes. Inaddition, questions for further study for future quality management efforts were identified. Although this qualitymanagement effort was for a spacecraft component production process, the ideas and approach presented can be ofbenefit to project managers across many fields and can be utilized on practically all quality management efforts.ReferencesBuckle, Rose, & Stuart, Ted J$.,$ Jr$.(1996,$ April $1).$ Systematic approach to patient waiting times. Physician Executive,American College of Physician Executives.Graves, Ray. $(1993).$ Total qualityDoes it work in engineering management? Journal of Management in Engineering,$9(4),444-455.$Juran J$.$M$.,$ & Gryna, F$.$M$.(1988).$ Quality control handbook $(4$th ed$.).$ New York: McGraw$-$Hill.Kuprenas, A$.,$ Kenney, M$.$D$.,$ & Nasr, E$.$B$.(1999).$ A Pareto analysis of construction and maintenance operationsaccidents. Second International Conference for the Implementation of Safety and Health on Construction Sites. Honolulu,Hawaii.Project Management Institute Standards Committee. $(1996).$ A guide to the project management body of knowledge$($PMBOK Guide$).$ Upper Darby, PA:Project Management Institute.

What is the background of the study