read this case 1

Project Management: AT&T Wireless Self-Destructs | CIO

Page 1 of 6

NEWS NEWS NEWS NEWS

Project Management: AT&T Wireless Self-Destructs

By Christopher Koch

CIO | Apr 15, 2004 8:00 AMPT

Jack Lee marked Nov. 24, 2003, on his calendar because that was the day he would finally

be able to change his cell phone carrier without losing his phone number-thanks to a

Federal Communications Commission ruling. But Lee, president of Tangara Technologies, a

company that develops software for forms, decided to wait a day before switching to AT&T

Wireless to let the chaos of "number porting" die down a little. Little did he know that the

chaos was just beginning.

Lee ordered his new phone on Nov. 25. When he went to the AT&T Wireless website to

check the status of his order a day later, he was greeted with a message: "We could not find

a porting request for this number in the system. Please contact Customer Care." It was the

beginning of a two-month odyssey in which Lee estimates he made 15 to 20 calls to AT&T

Wireless, sent nearly as many e-mails and spent 60 hours on the phone dealing with

customer service representatives or waiting on hold-with the line often going dead when

AT&T Wireless's customer service lines became overloaded.

After being routed all over the company, Lee finally discovered what was going on. A major

CRM system had crashed during an upgrade, and customer service representatives could

not set up or access new accounts. The system breakdowns, which continued through

February 2004, swamped other AT&T systems, gridlocked customer service phone banks

and sent furious customers scurrying to other providers.

The breakdown couldn't have come at a worse time for AT&T Wireless. It deprived the telco

of thousands of potential new customers and cost the company an estimated $100 million

in lost revenue. But that wasn't all. The failure so damaged AT&T Wireless's reputation that

many analysts believe it hastened its sale to Cingular in February for $41 billion, or $15 per

share, which was just under half the value of AT&T Wireless's shares when it went public in

April 2000. While an AT&T Wireless spokesman says the company would have been sold

regardless of the fiasco because "it was the right thing to do," the crash and resulting

http://www.cio.com/article/2439700/project-managementroject-management--at-t-wireless/...

9/8/2014Project Management: AT&T Wireless Self-Destructs | CIO

Page 2 of 6

confusion could not have helped AT&T cut a good deal. "The system problems made AT&T

look like a wounded provider, and the sharks smelled blood and circled," says Roger Entner,

a wireless and mobile services analyst at The Yankee Group, a research company.

AT&T Wireless's mistakes offer valuable lessons for CIOs. For one, it's unwise to freight

major system upgrades with external complications. AT&T Wireless's CRM upgrade was

hamstrung from almost the very beginning by rumors of outsourcing deals and future

layoffs. These rumors generated pervasive morale problems that hurt the productivity of

project staff.

Second, it should be understood that complex projects require flexible deadlines. AT&T

Wireless undertook a difficult upgrade that affected roughly 15 systems just before it was

faced with an immovable deadline-the federally mandated Nov. 24 number portability

date.

Finally, it always pays to have a plan B. Without one, AT&T Wireless was forced to move

forward even as it became apparent that its upgrade would not be completed in time.

Playing Catch-Up with the Competition

When AT&T Wireless began its Siebel CRM system upgrade in 2003, it was a company that

had slipped from unquestioned market leader to middle of the pack. Its overall market

share had slid from an industry-leading 25 percent at the end of 2001 to 17 percent in

2003, third behind Verizon and Cingular.

Worse, AT&T Wireless was playing catch-up on its most important technology asset, its

phone network. One of the older wireless companies, AT&T Wireless made an early bet on a

technology called TDMA that could not handle data transfer over cell phones-the next big

thing for business customers. Even before AT&T Wireless was spun out from its parent

AT&T in 2001, it had begun a furious buildout of an expensive new network-global system

for mobile communications, or GSM-that could not only handle data but had the added

advantage of global compatibility with overseas providers. Only one other major carrier,

Cingular, was saddled with the challenge of building out a new GSM network while still

servicing the old one. The other carriers had all chosen network technologies that could

handle data transfers.

GSM was a great opportunity for AT&T Wireless, but it was also a huge CRM challenge. The

company had to convince its old customers to move off TDMA, which worked as well as

most other carriers' networks for voice calls, and onto GSM, which had poorer voice quality,

http://www.cio.com/article/2439700/project-managementroject-management--at-t-wireless/...

9/8/2014Project Management: AT&T Wireless Self-Destructs | CIO

Page 3 of 6

according to Morgan Stanley. It also had to convince new customers that GSM was the

wave of the future, that they would soon be shipping data over their phones instead of

their laptops. But customers didn't buy the pitch. By 2003, AT&T Wireless's percentage of

customers on GSM was hovering at 15 percent, according to analysts, while Cingular had

35 percent of its customers moved over (thanks, in part, to its acquisition of a cellular

provider with an existing GSM network). And things weren't improving. In the third quarter

of 2003, less than half of AT&T Wireless's new customers were choosing GSM, while

Cingular was signing up 75 percent, according to Morgan Stanley.

Everyone at AT&T Wireless agreed that the company would keep its existing customers and

add more new ones if its customer representatives could handle more calls and get

customers up and running faster. Customer service representatives needed about 20

minutes, on average, to work through five or six screens that fetched information from

about 15 legacy systems, say former employees. Slow access to customer information

records was just one of the reasons why AT&T Wireless possessed the second-highest cost

per subscriber (behind Sprint PCS) of the top six national carriers in 2003, according to

financial services company UBS.

Why AT&T Needed to Upgrade

AT&T Wireless installed Siebel's CRM system in 2001 to be the front end of its customer

service process. The back end, however was a complex mishmash of systems, say former

employees. Telco billing systems, for example, were stuffed full of different rate plans and

arcane metering processes. Systems that tracked calls and set up new phone numbers

(provisioning) communicated with hundreds of thousands of different telephone switches

around the country and the world. To work for AT&T Wireless, Siebel's version 6 had to be

highly customized. Though the software came with integration tools, consultants usually

resorted to writing point-to-point scripts to hook the systems together. Policing the overall

integration in a scenario like this is difficult at best. Indeed, a former AT&T Wireless

employee who worked on the project recalls the test system crashing and remaining down

for six weeks during the summer of 2002 when AT&T Wireless began preparing Siebel

version 6 to deal with number portability. And when Siebel 6 was finally up and running, it

still couldn't handle all the information that customer service representatives needed.

The next release of Siebel, version 7, was much more powerful. Siebel developed an

industry-specific version of the new software that had many more features and could

capture more telco-specific information. And new Web capabilities meant that AT&T

Wireless could potentially reduce the number of customer service screens to one by

http://www.cio.com/article/2439700/project-managementroject-management--at-t-wireless/...

9/8/2014Project Management: AT&T Wireless Self-Destructs | CIO

Page 4 of 6

building a Web portal that put all the different systems and information sources together in

one place. "That was the point of the upgrade," says a former employee. "They wanted to

get everything on one screen so they could sign up customers faster."

Marc Siegel, a spokesman for AT&T Wireless, agrees:"As we needed to handle more

transactions, more customers, we needed a more robust system. So we upgraded. [The

upgrade] was going to make it easier for our representatives to get access to information

and give them a fuller array of information on the customer."

In the spring of 2003, the company decided to upgrade to version 7.5 for the roughly 3

million customers on the GSM and general packet radio service (the data portion of the

new network). TDMA customers would continue to be serviced through AT&T Wireless's

legacy Axys CRM system until they could be moved over to GSM and the new Siebel

system. The project was called Odyssey.

The Upgrade Begins, Badly

Though the upgrade promised gains in speed and simplicity, getting there was neither

speedy nor simple. AT&T Wireless's IT staff had to rip apart old links from the different

legacy systems to the client/server-based Siebel 6 system and rewrite them to work over

the Web. The back-end systems-integration work was so complex that it wasn't unusual to

see teams of 20 or more people assigned to write connections for a single system, says a

former employee. Coordination between the teams-the responsibility of the lead

integrator on the project, Deloitte and Touche-quickly got out of hand. (Deloitte and

Touche did not respond to repeated requests for an interview.)

"Everything was siloed among the different groups, and we all worked independently of

each other," says a project team member. Teams would work on a revision to their piece of

Odyssey, for example, only to find that when they finished testing, code had changed

elsewhere in the system, rendering the testing meaningless. "In other projects I've worked

on, the project managers would freeze code while the teams did revisions to their pieces so

you could test everything against the same code base," he says. "I didn't hear of that

happening on this project."

Meanwhile, rumors of layoffs and offshore outsourcing began swirling around Odyssey.

"[The rumors] slowed things down," says a former employee. "When stuff like that happens,

people start looking for other work. I know I was looking for other work when I should have

been testing."

http://www.cio.com/article/2439700/project-managementroject-management--at-t-wireless/...

9/8/2014Project Management: AT&T Wireless Self-Destructs | CIO

Page 5 of 6

Meet the New Boss

On April 10, 2003, Mike Benson, a 15-year AT&T veteran, sent a memo to employees

announcing that he was leaving after three years as CIO. AT&T Wireless's Siegel says the 48-

year-old Benson "decided to retire." Christopher Corrado, the head of the security solutions

practice for Wipro, an Indian offshore outsourcing company, was named executive vice

president and CIO. Before joining AT&T Wireless, Corrado had been CTO at two divisions of

Merrill Lynch, where he presided over the offshore outsourcing of portions of Merrill

Lynch's IT. Employees speculated that it was just a matter of time before offshoring began

at AT&T Wireless.

"Corrado was the hatchet man-everyone knew it," says a former employee. "We'd see our

people going into these long meetings with people from Indian companies. We'd see

whiteboards that had questions like, 'What opportunity do we have to

offshore/outsource?'"

Former employees say morale wasn't helped by Corrado's first presentation to the IT

group, in which they say he proclaimed, "Come in every day and expect to be fired."

Intended to inspire the troops to greater effort, the talk backfired, says another former

employee. "We all came away saying, 'Who is this arrogant jerk?'" Corrado could not be

reached for comment.

The Deadline Looms

As November approached, AT&T Wireless was one of the last industry holdouts opposing

the Federal Communications Commission's new rule on number portability, along with

Cingular and Alltel. The industry had succeeded in delaying the change since 1998 on legal

technicalities, and AT&T Wireless was counting on another postponement, say industry

sources. But on Halloween, the U.S. Court of Appeals for the District of Columbia Circuit

rejected the appeal to move back the FCC's Nov. 24 deadline. An AT&T Wireless spokesman

told Bloomberg News that the company was ready for porting.

It wasn't.

Things were going so badly with the Siebel CRM upgrade that there was talk on the project

of trying to roll back to the old Siebel 6 system before the porting date. "Two weeks before

launch there was rollback talk," a former employee says. But project managers hadn't

http://www.cio.com/article/2439700/project-managementroject-management--at-t-wireless/...

9/8/2014Project Management: AT&T Wireless Self-Destructs | CIO

Page 6 of 6

preserved enough of the old system to make it feasible. There was no plan B. Project

members estimated that the team needed two or three more months of bug fixing and

testing to get the system running reliably.

And the Siebel system wasn't the only thing AT&T Wireless needed to get working before

the deadline. Wireless number porting required new systems that had to be integrated

with the different carriers' CRM systems. Five of the six top providers outsourced the

administration of the number changes to TSI Communications and used software built by

Telcordia. But in June 2002 (before the other carriers had announced they were going with

TSI), AT&T Wireless chose its longtime software and administration provider, NeuStar.

Ultimately, this would create serious interoperability problems for AT&T Wireless.

AT&T Wireless defends its decision by saying that NeuStar was the most experienced

provider in the market, having worked on landline portability in the early '80s. It also says

NeuStar's software offering was more complete than TSI's at the time AT&T made its

decision.

ITEC 411: Project Case Set 1 - [25pts]

Individual Assignment

Read case papers 1-5 (see link to case papers in Bb / Content).

Evaluate and analyze the project cases collectively in terms of a project and project management.

Consider and discuss in terms of the 10 PM body of knowledge (PMBOK) areas: scope, schedule, cost,

communications, resources, quality, risk, procurement, integration and stakeholders.

1. 2. 3. Post your analysis of the 5 project cases in the Bb / Discussion Board / Project Case Set 1. [15pts]

Include the following in your discussion:

Did the projects meet scope, on time and on budget?

What went right with these projects? Why?

What went wrong with these projects? Why?

Identify at least 3 common issues/themes among these project

Identify at least 3 lessons learned for Project Managers

Review your classmates' analysis on the discussion board.