PAPERS OF THE AFSC SYSTEMS 1/ COMNfAND . - Apps.dtic.mil

uLzZOWw o030z 0 0wwaw053z 2 wLUan000

ZZ I-oz!0.:5454F-U7

zzQ0ZL000)00-Jw-0Z 0-03w00.111(0wzU) 00W IU)w550-z4uL, io0Z!Z DO44LI-C I00000*u00j

--r.ww00LLT- 0UoVU owU.(acK. wZ MaLL W0ULlu-Li Z ZWZj-IwILuw 56w

wIoo0WI1zw(.ZoM)wC0.0 o OZZIiiu.0-w C.-0 wcc(0* 7Cl)U.0000

wm00a CCJ)WW/)w(/ LLCl)ZL0cc wZ1)O2D:WzaMLLILLLJ0wQ0W 0CZ0wCaIZc0i LJ58L-10-C

U,)wcCl,zcocoUU)I/ mc im-.00007

CI060

z0c,b0zCC oChzU 0I-wcnLuWLWzC0-za00OZw 0mOmL610.0LL)0C00.0*0el

wmJ2iw4JZzzI-0*4w)CD62

zU0Z0.ZV)00uLLLLL(/)wOUwiQCz0ZOw00wLj iCLPzzLL0.ICl))ccc0Z63z0LU j

wLLn-0z0LL I-01-Zo0zWFczW(I)l Www10 0Jo 0wLuL.LA-WJ647

0I-)LLz000wz(0Few0LL0p-w-cIUOWU()wZOWOwW-IM 065wo.

LI0Cl,oL)0lzL 404ozCo co C)Jj0I-f(fUJWcb1 534L07.L ZZZu00W32wL ja.OJ.I4C.)Z)-LU Cwi.(nuI-0z-o1w w C - LL 0.LU) (41LI.waIxa.66(

xzwZ 1xoW -W LL j LUzzw0wM U0M 0EILLc-w wZ wm.W zLLLI-MzC-JwLL Z0 j00W(OI0Lixmzvi-j00m0 z WLL,7-67

LUJw-0-00LULUZoUo coo0U0WLDowi0ZC,z-zw.LUcUP(Z0cc o.w0a.0zJ 0wLwo.LUCA)LUJCC4J()LCCL4z.-oZ.0c0L jwa.-0(1)00LLLUw m0ncc0MLULU U0SipSUcn w8

LUzL0-J0.z uo-Ja-JAo3oL4 o00-0 LL JzcnzLU00L0*40LULLU WO00L0Z-wCUOLU OW ZLLO.z0 690

I-.UUUC/) DC0L)iI U z-.a:)5co'ML)L) UZw C)c,)zzo(ocnujUHz zL) Zr o. Z(L.0n WWWDCZWZ00.UCl)UWWW1 ZpIwOD0(0f 1)2C*-0Z0/U) LZL(f):3iu1a

ci,00zcc)wLUzwzEI2cc) W 0Zc0zcza0zo0 0mm mCa-P P x zo0C71c

z LLwDz-JxowmlWW000mw*0z72w727

w.0I. IzwwaQo01rDo4:73

wUz00zIDoo-wz0LUC) L0-02 z-jw-Lwwum0 Z Wz0 W w0zu)0.L0ZnU)P rn2D z 0 LLU )0ZjLL (1)Zcc0U ccuZ0(n U w 0 Ow0.0--WL0a.U)LL0ZrrwUNL)Z0LUM00a.WW I 2wCC C-0C/)0-, Zz00ZO(On3Zwiu )Z0-r a: -7zz ow00ZC)z , LZ)Z C LLLw0 Q00zi0-U) U0 20ZUa:LLj Z-- 0C0U)Q ooz0wI- (nLwnC/ c0I-0UZL-LV-0wJC/74 L) rwo -i(

(nIcc

Cl)0zwLrLLL00C'), cnO76

C)wwzj'U-(I) C mzWzU0LLIpwo0 -zzaczDwC cc,0nII IUIw

z0z00cm4Uz000 0r0U Z C) LUCl)000 p-C

jwnc6p-JC6LO0seLLCLLI-0cr0o) 0 0 ooa7900

C/)0LU-0CLU,U00mLL C O/)-J-oU) 00CL WZ o i0 3 0JcU L))M 0. .n Z Uor O0 WCL.0'to0cl803-LU'(4

I-0100(U)0o-jDUca LLz -jzwuMI.LI- 0u m 2 Z1ri00l LL0:)C. mP-O0m tItI08Wa.0 .WLL0.zw0cr1)uL LCAwcl0AwlaOWWWWUUiL

z0-jUw0lcmLC)z0 LcoLUML L -M,LUZ-J0zz4ZCD0- coo0(r)cr0CZ.)000UUOC).e82

I-Cl01-.- 1wX.L 0 z0cCl) OL. LLL1jlw0UwJUJcmClz00'0o IjWWzcIW L)0083

(1)00CCIzX,000(1)W0w00()Z0ZwJ wcE0u4OW%0

mzw00l85

0Z20o022ouw*UJUwu86uc

zC,l0C0I-.zz00-LUzLILw00ZpczCL)20CwowC0z cc3 (LzCL)-iOc) Z0a0jpLC870U

I-z0ChU0oCL

LIzzOW zD0zcI1)zCL000J-JLw z000030

AD A142 784PROCEEDINGS PAPERS OF THE AFSC (AIR FORCE SYSTEMSSYSTEMS DIVAERONAUTICALSTAND.(U)AVIONICSAFBROCOMMAND)0.OH DIRECTORATEWRIGHT-PATTERSONUNCLASSIFIEDC A PORUBCANSKYNOV 82-"IIIIIIII lllllllENlllIF/G g/2NL

5MICROCOPYRESOLUTION1102812.5TEST CHARTNTIOI,,.IIIIL

LUoU0.ILCr90ClU

C44z 0(1)4UCC0LU00wL0c0091

Coan)LL0CA0on j)0Lzo(irj01092

D0I. .-"093

LU9-

zUDwlwLAU00Cc. LL01-2Cal00iCcr01jDWLLLLIL0I0 CC0ZzLLLWz Cz0L 95

z zC',)wwj wLo0 )z .-JAw4000z096Lh4ww

.-00uLL-U.0UL041LUzz20C00ccULJUDLLI,LUJ0LLUUj-LLU7 D97j(C MI-l0

w0u0N-ozU)wULw0mwL60LDoWLL0owcci0 Cl)-oL JI-oZcnI-0OI2w OO wO Z (0wL00*098

D wcc0)wUUULLwL0w99-

000000z,-0u00000uo00I1000000' Z00U Ch0.L200 00.04L0LC.1904Zz0 a!U.00.0.100CEWm.000

wzw0U,0I-wzwo0ChJ0101

wUN00jw wZwLwZ ww102

C,)Ni,,7NcD-0zoouuinX ) Lunc0103L I.0CfL C/ccwu 4zL0lLLIWILJ-00)(Lu00

Ul)104

.w w:10iJujO mCCLL0OZw000uWI LUc105CCw0

U)10zzzLUmCCOoz -IccUJ'0C0Da.L106-9LL (3C0 0I

LU-J'0LU100

z02 U108

00000L0 ZwZzzzz0 00,000000000*0109000

I-wo0-I-0WI--0no110,i

1C(A0LU

ILLUojU.FzuCCXz-112LUIa

0oU(LUoLwCC5U4-AJ4uI-UUx4ClC)an40D

C,)LULULUco)0LU0CALUzoCCO0CCO LU0LUCoJI-zoam0CO

000Zzuz 1000a00w00115

LU0lzhF404zLU z20Cn0zCZ400.@000.u116

wCl)117

zoz0 U(LJzCl)F4)2'0'- Wcr ZF WWCCn1-0aLuWc0LUj0 Zo OzLft)azI-J0LJwr0LU -00cL00.Jcc0 a.0 x0Oa.- CLU00 (0C/) lCC UL118

IIwzU.00w0zchzI-Ec0C,,a0WOF00ZLLU4j0ca.0 -1oLwCl00 04hW U0C@0119cc,-.zCc/0U-3w.0 z - cn d@0z

WClI.-I-00wwXCA0w-ui wZL)LwU0.-0WCl)ccw0owSQ0ml0C)L.-z zZwl0 000wa0mw1 2000

uiI-Cl)cfCO)zwL0 m000 Ci)z0 j0zlI-N Iwm 0I-JaI-(WUcc 0D0-ZDf0WmccJCMwL)LAU. 0Wuwa.IUin IE121 (n UIa0

Cz w40000.00LU02Lzicz Lucrr OrCl znwU)U0122Lc

0cc:w-criwULLJ V)a.w(I)crxacc ChzwLZ-0wj 0L000zF ZOU 0amJJ(W0 10Cl)123z0

a0045W0ILi LL.O I-*0 z&coW0(0I0U0,0EoDw0Ux1L) zmw-JJL)0u.ZWOr0UC1L10-Izm00LULwoLLL0 0z-.2Iza.LCC wjI ZL.ICL24zI

SPECIFICATION SHEETNAVAL ELECTRONICS SYSTEM CASE STUDYTOPIC:Management problems and solutions associated with a Naval electronicsembedded computer software system throughout its development history.TYPE:Case Discussion,1 1/2 Hrs.OBJECTIVES:. To provide the participant with an exercise in analyzing embeddedcomputer software management activities.* To provide the participant with an opportunity to apply his own skillsand experience to a set of typical problems associated with virtually everycomplex software intensive system.* To provide the opportunity for participants to trace, in a singlethread fashion through the entire development cycle of a system as itexperiences early difficulty, periods of constant reassessment, and finally,success.125

NAVAL ELECTRONICS SYSTEMDISCUSSION:In the fall of 1974, the Navy Project Management Office - Electronics(PME) solicited an RFP for proposed system designs for a naval electronicssystem. Of the original 38 contractors who attended the initial briefing,four teams of contractors submitted proposals.Two of these contractor teamswere eliminated, primarily due to the fact that their proposals disclosed thatthe contractors did not understand the depth and complexity of the requirements. The remaining two contractors were selected to submit system specifications by April of 1975. At that time, two contenders were placed under cadretasking while their proposals were evaluated.Under the cadre tasking, thetwo contractors were directed to further refine their specifications and toIn August ofdevelop preliminary Program Performance Specifications (PPSs).1975, the PME selected one company as the prime contractor with a separatecompany as the major software developer.The software subcontractor wasdirected to continue development of the PPS and the Program Design Specification (PDS).The contract for full scale development was signed in March of1976.Under the terms of the contract, the software subcontractor was requiredto utilize a top-down modular design methodology, the CMS-2 high orderlanguage, and structured programming techniques.As top-down implementationof the design proceeded, the software subcontractor would deliver software infour basic increments resulting in a final delivery of an integrated, testedsoftware system by May 1977.System integration of software to hardware wasThisspecified for completion, with acceptance testing, by September 1977.allowed a period of only 18 months from the start of full scale development tocompletion of acceptance testing for a system composed of 18 hardware racksand associated system software; a very ambitious contract.The PME, realizing that they had a lack of computer software trainedpersonnel, negotiated with the Naval Ocean Systems Center (NOSC) to providesoftware support during the development phase of the program.NOSC is a majorNavy laboratory that is electronics-oriented. They experiment in microelectronics, radar, and satellite systems.On this program, NOSC was specificallytasked to:1. Provide support to the PME during the review and evaluation ofcontractor produced software and documentation.2. Serve as the single point of contact in the provision of softwaresupport to the prime contractor.3.Provide facilities and instruction to the computer system hardwareand software maintenance agent, the Naval Electronic Systems EngineeringCenter (NESEC).4.Install Level 2 Support Softwarecontractor development and test facilities.(CMS-2M,SDEX120)5. Perform testing and verification of incrementaldeliveries and report results to the PME.1261]at thesoftware

In support of these requirements, NOSC would partake in all design andprogram reviews, computer program regeneration, and computer programacceptance demonstrations.NESEC, San Diego was designated as the software maintenance agent.It wasrecognized that NESEC would have to participate in all phases of developmentto gain the necessary experience to undertake software support functionsupon completion of system development. NESEC personnel were to interface,primarily witn NOSC, to gain the needed hands-on experience.During the eleven-month cadre tasking period, the software subcontractorcontinued to refine the system specifications and develop the PPS and the PDS.A preliminary PPS was delivered just prior to the signing of the full scaledevelopment contract.Within 90 days of the contract signing, a preliminaryPDS was delivered.No realistic new cost or schedule reestimation wasattempted.Upon delivery, the PPS and the PDS were submitted to exhaustive designreviews. The PPS document was determined, with minor exceptions, to accurately specify system requirements but was not in the format required bySECNAV Instruction 3560.1.Rewriting of the PPS. in accordance with theinstruction, resulted in a six-month delay in delivery and acceptance of thefinal document with corresponding delay in placing the allocated oaselineunder configuration management control.The initial version of the PDS constituted well over 1000 pages and was subsequently determined by the softwarecontractor to specify design requirements at too low a level and in too greatdetail.The final document was on the order of 200-300 pages.A significantshorLcoming of the PDS was that it did not provide detailed interfacespecifications between software and hardware.The programming phase of software development commenced upon initiationof the development contract. The software subcontractor began coding beforeeither the PPS or the PDS were finalized.In July of 1976, it became apparentthat the software subcontractor was not spending funds in accordance with theCost and Schedule Control Program (as required by DODD 7000.2).This initiated an investigation that indicated that the software developer was notmeeting the scheduled requirements for delivered lines of executable code.Although reluctant to admit to development delays, the software subcontractoreventually had to acknowledge that they were experiencing significant softwareproduction difficulties when they missed delivery of the second softwareincrement in November of 1976.In December of 1976, a number of actions wererecommended to contain cost growth and to schedule a slip in the program.Among the actions taken were replacement of an IBM 370/135 support computerwith an IBM 370/145, partial delivery of the second software increment, andsignificant reorganization of the software subcontractor management.At thistime, software development was from three to four months behind schedule.With the partial delivery of the second increment of software in FebruaryIt soon became apparentof 1977, integration of software to hardware began.that significant problems existed in accomplishing the integration because ofinadequate implementation of interface controls within the software. As integration difficulties expanded, more and more resources were diverted to thetest site to contain the problem.In order to allow more time to deal withintegration problems, the hardware development began to drive the software.127

Even though software development was three to four months behind, the NavyThis resulted in abandoning incrementalrefused to relax the schedule.deliveries in favor of drop deliveries of software modules which would interface with the emerging hardware devices. This required that the softwaresubcontractor reestablish software priorities to support hardware availability. The change from incremental software deliveries to drop deliveriesseriously impacted verification and the test center's ability to provide testresults in a timely manner.At the same time, a problem of simple logistics became evident. The 400mile separation between the test site and the software development personnelwas creating additional delays in development.As a result, in December 1977,all software development personnel were relocated to the test site to optimizethe integration process.While the major software development was going on by the software subcontractor, NOSC was expending considerable resources in establishing an independent verification and test center at NESEC, San Diego.A test facility,consisting of operator consoles and AN/UYK-20 computers with supporting peripherals, was set up.Test drivers which simulated the various system hardwaredevices were written.Software increments were simultaneously delivered bythe software subcontractor to NOSC and to the main test site at the primecontractor's facility. Various tests conducted on software delivered by thesoftware subcontractor uncovered errors, especially in modules which interfaced to hardware.There was, however, no correlation performed with errorsdiscovered at the main test site so it could not be determined whichproportion of errors discovered at NOSC accurately reflected errors which werecaused by improperly coded test modules.Although NOSC's test facility wassimilar to the system configuration in computer and peripherals, the front-endwas software simulated.Errors discovered in the system were often duplicatesof errors uncovered by the prime contractor. Front-end errors were oftenattributed to the software simulation at the San Diego test site.Although the San Diego test center did perform verification functions, t',eprimary purpose of the center was to provide training to NESEC personnel.Dueto the high priority and expedited schedule of the program, there was not theusual three to four year test phase to gain knowledge.Therefore, the testactivities at NOSC provided invaluable experience for the software maintenancepersonnel from NESEC.At the present time, system deploynent is on schedule.The first systemwas fielded three years following the start of full scale development.Thesoftware subcontractor is under a maintenance contract to help clear "bugs"and provide training to NESEC personnel. Although no changes were allowedduring the initial production due to the tight schedule, approximately 100class II changes were presently in work in 1978. Most of these changes are toimprove efficiency by revisions to the display format, timing, etc.In thesummer of 1979, the Navy assumed configuration control at the code level.Alllibraries were removed from the prime contractor and assumed by the Navy.CONCLUSIONS:Given the scarcity of formal guidance and the lack of computer softwaretrained personnel, the PME did a commendable job in advance planning and128

utilization of coinputer resources for the project.State-of-tne-art designand developnent techniques were demanded of the contractor such as new hardware, new support software, top down structure, and thie latest USN standardsand specifications.The need for a maintenance support agency for postdevelopment pnases of tne software lifecycle was recognized early.Foresightwas evident in insuring that the maintenance agency would receive adequatetraining ano participate in all aspects of development.Acknowledging theirlack of software expertise, tne PME employed NOSC to assist in providing forquality assurance of the delivered product.The use of a high order language,structarea programinng, and good program documentation was specified toimprove ease of software maintenance.Personnel from the user comnmunity weretrained and incorporated into the test program.Problems in production weredetectei early and aggressively attacked by the project management group.Unforturat ely, advance planning and close program monitoring by the P14E werenot suffcieitto prevent a significant software cost overrun and schedules1 ippae .'e two Predo-inant causes of cost overrun and schedule slippage wereundero.stiniat on of the tasK complexity by the major software contractor andinadequate Mianning for the system integration by the prime contractor.Discussions with the software subcontractor disclosed that they themselvesfelt that too con)iexity of software requirements was grossly underestimateddue to the uticationof a relatively naive engineering team to initiallyscope -tne procief a.c design the system.Specifications resulting from tniseffort were not aaequrately reviewed by higher level management within thesoftware suOco-tr)tor organization and were not oalanced against preliminarycost. and time estvmaions.Software subcontractor management admitted tiatthis couls nave been accomplished by project management personnel if adequatefunds and time had been provided.The assumption that the underestimation ofprogran cnp exi' y was cmain problem is borne out oy 3rot!: in source -odeand by code prd ction figures for the project which show that lines of codeproduceu per ,ian irwere slightlyabove average for the industry.It canalso be shown that the software subcontractor's original budget for cost perexecutable Lineof code called for a production rate well above the norm.Inspite of tr.ese proobiens,

Volume 4 MIL-STD-1553 Tutorial Volue 5 MIL-SD-1589 Tutorial Volume 6 MI r-S'TD-1679 Tutorial Volume 7 ML-STD-1750 Tutorial Volume a mI-STD-s 1815S Tutorial volume 9 Navy Case Study Tutorial PROCEEDINGS OF THE 2nd AFSC STANDARDIZATION CONFERENCE 30 NOVEMBER - 2 DECEMBER 1982 DAYTON CONVENTION CENTER DAYTON, OHIO Sponsored by: Hosted by: