mopdf.com

Audit ResultsAudit BackgroundThe Federal Acquisition Streamlining Act of 1994. The Federal AcquisitionStreamlining Act authorized simplified acquisition procedures for procurementsup to 100,000 using Federal Acquisition Computer Network (FACNET).FACNET is a communications network that the Government uses toelectronically transport Electronic Commerce/Electronic Data Interchange(EC/EDI) transactions between Government buying organizations and theirtrading partners. The FACNET infrastructure is composed of buying activities,gateways, network entry points (NEPs), value added networks (VANs) andtrading partners. A buying activity sends a transaction through an applicationsystem to the supporting gateway. After translation, archiving, and otherfunctions are performed by the gateway, it transmits the information to a NEP.The NEP receives the EDI transactions and transfers them to VANs that havebeen certified. VANS distribute the information to trading partners. Tradingpartners return EDI transactions to the buying activity in the reverse process.The goal of F ACNET is to speed up procurements and reduce procurementcosts through increased competition.Responsibility for Supporting FACNET. In a memorandum, "DefenseElectronic Data Interchange Infrastructure Implementation," June 23, 1995, theAssistant Secretary of Defense (Command, Control, Communications, andIntelligence) assigned the Defense Information Systems Agency (DISA)responsibility to manage the FACNET infrastructure. DISA established theDoD EC/EDI Customer Service Center (CSC) Help Desk at the Ogden, Utah,Megacenter, to respond to problems encountered by participants in F ACNET.Users of FACNET can report problems or questions to the CSC using a toll-freeline 24 hours a day. If a question or problem cannot be resolved immediately,the CSC generates a trouble ticket. Six customer support personnel at theMegacenter review and analyze the trouble tickets. The CSC receives about294 calls per day; however, customer support personnel at the Megacenter didnot know how many of those calls were trouble ticket calls. The CSC generatesabout 14 FACNET trouble tickets per day. As of March 1996, there were971,632 EC/EDI transactions processed through FACNET. Those transactionsincluded 164,615 public transactions, which are requests for quotes, quotes, andpurchase orders. The remaining transactions included, for example, testtransactions, functional acknowledgments, delivery orders, or sole-sourcerequests for quotes not advertised to the public.2

Audit ResultsAudit ObjectivesThe primary audit objective was to examine the effectiveness of the troubleticket process and to identify problem areas that, if corrected, would result infewer trouble tickets. We also reviewed the adequacy of the managementcontrol program as it applied to the primary audit objective. See Appendix Afor a discussion of the scope and methodology and the review of themanagement control program. Appendix B summarizes prior coverage relatedto the audit objectives.3

Recurring Problems That Result in Trouble TicketsDISA has not resolved recurring problems identified by the trouble ticketprocess because resolution will require a major redesign of the existingFACNET or implementation of a new system. Recurring problems withthe FACNET include:o invalid transactions due to data input errors and incompleteinformation on transactions,o lost and late transactions resulting both from unreliablecommunications links between the gateways and the VAN s and frominoperative gateways,o an inability to track transactions as they flow throughFACNET, ando lack of acknowledgments for receipt of transactions uponarrival at their destinations.As a result of the recurring problems, users are dissatisfied with theFACNET trouble ticket process, trading partners have filed protestswhen bids have arrived too late to be considered, and the future successof the F ACNET trouble ticket process is uncertain.Method of AnalysisWe selected and analyzed a statistical sample of 130 trouble tickets from2, 163 trouble tickets received at the Ogden Megacenter during June throughNovember 1995. Our analysis identified a series of problems that fall intoseveral categories as shown in the figure on page 5.4

Recurring Problems That Result in Trouble TicketsLate and/or LostTransactions(29.0%)Other Problems(38.0%)Invalid Transactions(33.0%)Types of Problems Identified by Trouble Ticket SystemWe categorized the trouble ticket problems as invalid transactions, losttransactions, late transactions, and other problems. Other problems consisted ofstatus of receipt, software, modem problems, procedures, and vendorregistration. Three categories of recurring problems resulted in the mostnumber of trouble tickets, specifically, invalid transactions, lost transactions,and late transactions. Additionally, although not specifically identified in oursurvey, we regard the inability for DISA to track FACNET transactions as arecurring problem because FACNET users frequently contacted the troubleticket desk to inquire on the status of FACNET trouble ticket transactions. SeeAppendix C for detailed results of our analysis of the sampled trouble tickets.Types of Problems Identified by Trouble Ticket SystemInvalid Transactions. Forty-three trouble tickets were related to data inputerrors and incomplete information on transactions. For example, one of theVAN s reported that the DoD gateway was sending certain transactions to anerroneous receiver identification number.Lost and Late Transactions. Thirty-eight trouble tickets involved procurementtransactions that were sent but never received by the intended party or were5

Recurring Problems That Result in Trouble Ticketsreceived later than the sender expected. These involved seventeen vendors.Personnel at the VAN s informed us that many of the lost and late transactionswere the vendors' bids in response to requests for quotes. Personnel at theVANs identified 38 vendors who reported lost or late bids. Eleven of the38 transactions were lost or not received by the buying activity until aftercontract award.The lost and late transactions were attributed to several causes, includingsoftware problems at the Gunter Air Force Base Gateway, which causedmessage files to be lost or overwritten, and inoperative equipment atDoD gateways.Status of Receipt. Four trouble tickets involved inquiries as to whether atransaction had been received by its intended recipient. The American NationalStandards Institute (ANSI) standard X12, on which FACNET is based, requiresthat a receipt, known as a functional acknowledgment, be automatically sent tothe originator of a transaction upon receipt of the transaction. This processworks much like sending a registered letter through the Postal Service. Withoutthe functional acknowledgment, the trading partner has no proof that atransaction was received. Users who reported problems to the CSC that resultedin trouble tickets complained that FACNET did not always generate a functionalacknowledgment for receipt of a transaction. Users also contacted the CSCwhen they did not receive a functional acknowledgment after submitting atransaction through FACNET.We analyzed the functional acknowledgment transactions. Our review of the130 transactions in the sample showed that there were no functionalacknowledgments for 115 of the 130 sample transactions. Of the 115 caseswhere functional acknowledgments were not received, 15 were requests forquotes. The causes were break down of modems, and failure of computerequipment and software at the Columbus and Ogden NEPs and at the GunterAFB Gateway.Inability to Track FACNET Transactions. FACNET has no automatedcapability to track transactions through the network system. As a result,F ACNET users could not readily determine whether their transactions werereceived in time for contract award. When users inquired about specifictransactions at the Ogden Megacenter, the CSC generated a trouble ticket andmanually tracked the transactions. This process is time-consuming and oftendid not permit the user to resubmit the transaction, if necessary, in time foraward consideration. The inability to track F ACNET transactions directlyaffected DISA's ability to provide a timely response to users who inquired aboutlost and late transactions or whether a transaction had been received.Other Issues ReportedUsers identified other problems related to the vendor registration, proceduresfor resolving problems, modems, software, and status of receipt of transactions.6

Recurring Problems That Result in Trouble TicketsRegistration. Thirteen of the sampled trouble tickets related to vendorsand organizations that were not registered in F ACNET or vendors that hadregistered but provided erroneous information when they registered. Thosevendors and sites were not able to process transactions through FACNET.Procedures. Twelve of the sampled trouble tickets related to users'questions concerning procedures employed to resolve certain problems.Modems. Eleven of the sampled trouble tickets related to modems thatwould not allow the customer to send transactions through F ACNET.Software. Nine of the sampled problems related to communicationssoftware that could not transmit data to FACNET.Status of Receipt of Transactions. Four of the sampled trouble ticketsresulted from users' inquiries about the status of transactions.Issues Related to Recurring ProblemsThe recurring problems are attributable to a series of issues including the shortstatutory time frames for the implementation of F ACNET; lack ofstandardization; allowance of broad participation by VANs; and the use byDISA of slow-speed, unreliable modems for linking VANs to FACNET.Short Time Frame. The Federal Acquisition Streamlining Act of 1994required that FACNET be implemented by January 1997. The milestone didnot permit sufficient time for DISA to properly test FACNET or to procure theproper hardware and software. Consequently, DISA used existing computersfor NEPs and gateways that were not well suited for F ACNET technicalrequirements.For example, the NEP computers encountered technicaldifficulties when running multiple tasks and were not always reliable whenpolling the gateways for transactions. Further, because the computer vendorwas experiencing financial difficulties, it could not readily correct the problems.As a result, transactions sent through FACNET were lost and late and not easilytracked, which resulted in the reported trouble tickets.Lack of Standardization. Trouble tickets relating to invalid transactions weredue in part to the use of an inappropriate version of the ANSI X12 standard forthe transactions by vendors and VANs. DoD is using multiple versions of the3010 ANSI X12 standard. The use of various versions of the ANSI X12standard increased the chances of errors in data input and translation of the datato and from the ANSI X12 standard. In addition, the use of the multipleversions confused FACNET users because the information required for eachversion is different. Transactions generated for one version of the ANSI X12standard are not directly interchangeable with other versions and requiretranslation by special data mapping software. Multiple versions of the ANSIX12 standard caused invalid transactions, which could not be sent to and fromvendors until the transactions were corrected and resent through FACNET.7

Recurring Problems That Result in Trouble TicketsBroad Participation by VANS. To encourage part1c1pation in FACNET,DISA approved and certified multiple VANs to participate. As of April 1996,26 VANs were certified.However, Inspector General, DoD, ReportNo. 96-172, Certification and Management of Value-Added Networks,June 24, 1996, reported that 15 VANS with questionable financial resourceswere certified, and the Government was not able to ensure that control wasexercised to prevent and resolve deficient services by the VANs. Each VANhad unique connectivity and data processing requirements that interfered witheffective transmission of transactions through FACNET. The VANs reportedproblems of invalid transactions and unreliable communication links togateways. The unique connectivity and data processing problems by the VANsresulted in interruptions in the transmission of transactions, lost transactions,and late transactions. Also, unreliable communications links between thegateways and NEPS made it difficult to track transactions as they flowedthrough F ACNET.In addition, customers did not receive functionalacknowledgments from buying activities, gateways, NEPs and VANs, whichwould identify the location of transactions in FACNET.1111Unreliable Modems. DISA allowed the gateways and NEPS to use unreliablemodems to connect to FACNET. It would not have been cost-effective toupdate the current modems with imminent redesign of FACNET (see the sectionentitled DISA Efforts to Resolve Recurring Problems later in the report fordetails on the redesign). The modems did not always connect properly toFACNET, thus requiring manual intervention by DISA to keep them operating.As a result, 38 transactions out of 130 sampled were lost or late.1111Effects of Trouble Ticket IssuesFACNET Success. For FACNET to be successful, users must have confidencethat it can deliver procurement transactions to intended recipients in a timelyand cost-effective manner. As a result of the recurring problems, FACNETusers, both within DoD and vendors, indicated a high level of dissatisfactionwith the trouble ticket process and with F ACNET.Consequently, thecredibility and future success of FACNET is questionable.Users' Feedback on Trouble Ticket Process. Results from the analysis of thetrouble tickets showed that 47 trouble ticket users, including vendors, were notsatisfied with how DISA handled their problems. A total of 46 FACNETtrouble ticket users we surveyed stated that problems were not resolved. Some104 users, which included 100 VANS and 4 vendors said that reported problemshave been the same types of problems reported by them on previous occasions.One hundred and three of the customers acknowledged they receivednotification from the NEPS of the status of the problem reported on theirtrouble tickets. One hundred and twenty three of the customers received atrouble ticket number from the CSC when they called back to ask about theirproblem.8

Recurring Problems That Result in Trouble TicketsDISA has made significant progress in processing the backlog of trouble tickets.During June through November 1995, the number of trouble tickets reportedper day was about 14, the average time to process a trouble ticket was 23 days(ranging from 1 to 153 days), and the backlog of tickets was 578. The OgdenNetwork Administrator stated that as of March 22, 1996, the backlog decreasedto about 76 trouble tickets. He stated that the number of trouble tickets reportedper day is still about 14, and the average time to process them was still about23 days. (Paradoxically, figures reported by the Office of the Director, DoDElectronic Commerce, indicate that the average time to process a trouble ticketas of March, was 6 days.)Officials at Simplex, Incorporated, a certified VAN, stated that problems wereoften resolved too late, if ever. Requests for quotes placed on FACNET allowfor responses to be made within 15 to 30 days; 29 of 130 transactions requiredmore than 30 days for resolution. Also, 21 of 130 transactions or 16 percent ofthe transactions were lost, late and invalid transactions, which resulted in lostopportunities to bid. A delay of an average of 23 days for resolution of a lostor invalid transaction is not acceptable, and has resulted in lost business tovendors. Because of the delay in trouble ticket resolution and repeatedproblems, users have lost confidence in F ACNET and are reluctant to use it.Vendor Protests. At least three vendors have filed protests as a result ofFACNET related problems. For example, one vendor protested the award of acontract to another vendor through FACNET by the Red River Army Depot.The vendor maintained that a bid was sent over F ACNET in time for the bid tobe considered, but the contracting office did not receive the transaction untilafter the award was made. The vendor who reported the trouble ticket problemwas the low bidder. The bid was received late through FACNET, and thelowest bidder did not receive the award.The Walter Reed Army Medical Center received two protests citing delayedreceipt of bids. In a memorandum to the DoD Director, Electronic Commerce,the Deputy Assistant Secretary of the Army for Procurement cited 36 otherinstances in which transactions arrived too late to be considered for award.However, the Army successfully resolved these instances without protests beingfiled.DISA Efforts to Resolve Recurring ProblemsProblems will occur with the implementation of any complex electronic-basedsystem. DISA established the trouble ticket process to identify specificproblems with FACNET and resolve them. As discussed earlier, DISA hasmade progress in processing trouble tickets, and the backlog of trouble ticketshas improved from November 1995 through March 1996. However, thenumber of trouble tickets reported per day, the time it takes to close a troubleticket, and the recurring problems being reported have not decreased. Inaddition, followup phone calls indicated that vendors are not reporting troubleticket problems, because they have lost faith in the trouble ticket process.9

Recurring Problems That Result in Trouble TicketsDISA Awareness of Issues. DISA is aware of the recurring problemsidentified by the trouble ticket process and is working to make long-termchanges that should resolve some of the recurring problems identified in thisreport. However, DISA has not resolved recurring problems in the short-term.System Redesign. DISA is redesigning the FACNET infrastructure andbelieves that the redesign should eliminate lost and late transactions, theinability to track transactions, and should provide functional acknowledgmentsfor transactions. DISA officials stated, however, that the redesign may notresolve problems relating to invalid transactions because those problems seemedto be related to user errors and data translation problems that are outside thescope of DISA responsibility.The FACNET redesign, referred to as the Electronic Commerce ProcessingNode, includes new equipment and software costing about 2. 7 million.According to DISA officials, the Electronic Commerce Processing Node shouldinclude a transaction audit trail, a functional acknowledgment summary, and anautomatic retransmission of messages when F ACNET identifies a transmissionerror. These changes should facilitate more expeditious tracing and resolutionof lost, late, or invalid transactions and should provide functionalacknowledgments.Implementation of the FACNET Redesign.We agree that theredesign, as proposed, should reduce the recurring problems related to lost andlate transactions, the inability to track transactions, and the lack of functionalacknowledgments for transactions. DISA is testing the new system; but has notestablished milestones for its implementation. Until the new system is in place,the recurring problems identified in the report will continue. To regain users'confidence DISA needs to establish when the new system will be implementedand functional. On August 5, 1996, DISA provided a report to the Director ofDLA informing him of the results of the DoD test of the Electronic CommerceProcessing Node. The report stated that The Electronic Commerce ProcessingNode delivered more than 650,000 transactions without any observed problems;demonstrated 100 percent accountability, and 99. 5 percent transactions werecompleted. Average speed of service was 58 transactions per minute under atraffic load of 50,000 transactions per day. The final test results were briefed toDeputy Under Secretary of Defense (Acquisition Reform/Electronic Commerce)on August 20, 1996. DISA was determined not to implement the ElectronicCommerce Processing Node capability on August 20, 1996, to ensure nodisruption of service to the contracting users during year end close out.In addition, DISA needs to identify how the recurring problems will be resolvedin the interim. Such efforts should both reduce the trouble ticket work load andimprove user confidence in FACNET.10

Recurring Problems That Result in Trouble TicketsRecommendations, and Management CommentsRedirected Recommendation.As a result of management commentsrequesting redirection of recommendations, we redirected Recommendation 3 .to the Deputy Under Secretary of Defense (Acquisition Reform). DISA statedthat the Deputy Under Secretary of Defense (Acquisition Reform) shouldmandate to the Services and agencies that functional acknowledgments arerequired for all transaction sets since the Services and agencies own theautomated information systems at the user ends.1. We recommend that the Director, Defense Information SystemsAgency, establish milestones for implementation of the redesign of theFederal Acquisition Computer Network infrastructure to provide promptcapability to fix the recurring problems identified in this report.Defense Information Systems Agency Comments. DISA concurred, statingthat the Department tested the new

FACNET trouble ticket process, trading partners have filed protests when bids have arrived too late to be considered, and the future success of the F ACNET trouble ticket process is uncertain. Method of Analysis . We selected and analyzed a statistical sample of 130 trouble tickets from 2, 163 trouble tickets received at the Ogden Megacenter .