Western Digital: Nearly All NAND Capacities Resumed Normal Operations
by Anton Shilov on August 1, 2019 4:00 PM EST
Western Digital and its manufacturing partner Toshiba Memory Co. (TMC) had managed to resume normal operation of almost all of their joint production lines at their Yokkaichi Operations campus in Japan, Western Digital said on Wednesday. Damages to wafer and manufacturing tools will cost Western Digital up to $339 million in total.
A 13-minute unexpected power outage in the Yokkaichi province in Japan on June 15 affected the manufacturing facilities jointly operated by Western Digital and TMC. The incident damaged wafers that were processed and also production equipment used by the companies. Western Digital said in late June that the accident would reduce its NAND flash wafer supply in Q3 by approximately 6 EB (exabytes), which was believed to be about a half of the company’s quarterly supply of NAND. Toshiba also confirmed that wafers and equipment was damaged, but did not elaborate.
By now, virtually all production capacities at the Yokkaichi Operations are back online, according to Steve Milligan, chief executive of Western Digital.
“Western Digital and TMC teams have worked diligently on recovery activities and as of now, nearly all of the equipment in the fabs has returned to normal operations.”
The company believes that all the lost wafers will be contained in the September quarter, but the incurred damages will be quite vast. In Q4 FY2019 (Q2 C2019) the company took a $145 million charge for impacted equipment as well as operations, and plans to take another $170 – $190 million write-off in the September quarter. As a result, the impact on Western Digital will total $315 – $339 million.
Being a private company, Toshiba Memory does not disclose the impact of the accident, but if the company lost the equal number of wafers and has had to restore its production capacities, so its losses will be comparable to those of Western Digital. Overall, the 13-minute power outage will cost the two companies $630 to $678 million.
Related Reading:
- Toshiba & WD NAND Production Hit By Power Outage: 6 Exabytes Lost
- Toshiba Memory and Western Digital Open Fab 6 and New Memory R&D Center
- Toshiba Memory & Western Digital Finalize Fab K1 Investment Agreement
- Toshiba Begins to Construct New BiCS 3D NAND Fab in Iwate Prefecture
- Toshiba Memory to Build New Fab to Produce BiCS 3D NAND
- Toshiba Finalizes Plans for New 3D NAND Fab: Coming Online in 2019
- Toshiba to Build New Fab to Produce BiCS NAND Flash
Source: Western Digital
Facebook
Twitter
RSS
30 Comments
View All Comments
Sivar - Saturday, August 3, 2019 - link
Manufacturing semiconductors can take months from start to finish. Anything being processed in equipment that cannot restart (most) will be either lost or, depending on the step, need rework (e.g. clean off the partial layer with acid then reprocess).The number does seem large to me, but it may be a very high output fab with a very large amount of WIP.
oRAirwolf - Thursday, August 1, 2019 - link
With so much potential for revenue loss and time required to recover, I am curious what kind of power redundancy they have. I understand this facility probably uses an image amount of energy, but I assume all of the power redundancy infrastructure necessary to keep the plant running would cost less than $339,000,000.00quorm - Friday, August 2, 2019 - link
This was covered pretty extensively in the comments last time an article about this incident was posted, but at some point it's not feasible to build additional backup power and they just get insurance instead. Assuming insurance pays out, they won't lose money, and may actually benefit if this event causes NAND prices to increase.Kristian Vättö - Friday, August 2, 2019 - link
There is no insurance for fab outage.mgs - Friday, August 2, 2019 - link
No sane insurance company would insure for this.DanNeely - Friday, August 2, 2019 - link
If you're willing to pay enough to a boutique insurer you can get a policy for almost anything. After a certain point though if the risk is too hard to price they'll end up asking so much that it's not worth buying a policy vs accepting all the risk yourself. (When in doubt they err on the side of not taking a major loss vs only earning their average return.)Skeptical123 - Saturday, August 3, 2019 - link
I have not read up on this incident heavily but I view your logic as reasonable except for one key point. I find it hard to believe the fabs used enough power to make having backups last longer than a few minutes unjustifiable cost-wise. The biggest expense cost-wise for backup power is an immediate supply of power to safeguard against grid fluctuates. Even if the grid has an issue for less than a second it can wreak havoc with the billions of sensitive manufacturing machines. As such, no company in their right mind would run such a plant without a battery backup (batteries and supercapacitors are the only sources fast enough to react in these senouries) in place. Of course the issue is that a batteries to run a plant that uses power in the megawatt range is prohibitively expensive for more than a few minutes. Which is way battery systems are normally designed to last only long enough to get back up generators up and running. For big diesel generators this can take a few minutes. A quick google search shows new portable 1 mega watt portable generators (in semi trailers) in the US cost about $400k. So the cost for ~20+ megawatts in the form of stationary backup diesel generators should cost under 10 million usd. Making it one of the cheaper if not cheapest systems in the fab.voicequal - Monday, August 5, 2019 - link
Generators are dirt cheap compared to the losses incurred here. No one would build a plant without at least enough back up power to perform an orderly shutdown, which I assume could take hours. They almost certainly had generators, but obviously some fault prevented them from operating correctly.dgingeri - Saturday, August 3, 2019 - link
My bet is they had the infrastructure in place, and it got old, and while executives argued about paying for replacements for this part or that part, the power outage occurred and someone finally realized they really needed the parts.This happens in corporations a LOT. Executives argue and debate about paying for things without realizing how much those things are really needed. Such penny pinchers don't realize the damage they do to themselves.
My current company has executives arguing against replacing our 7 year old storage. We had 8 drives fail on one unit within one day, causing our biggest customer to lose 7 VMs on one datastore. It took us a week and a half to get the storage back up and the VMs restored from backup. Our customer was extremely ticked off. That encouraged them to actually move that customer onto newer storage, but most of our customers are still on the old stuff, and we have 6-9 failed drives per week on it. We've already had two major customers leave because of problems from failed equipment.
That doesn't even get into the problems created from stressed employees because of lack of proper staff or paying well enough to attract staff with enough skill to do the job.
Asurmen - Saturday, August 3, 2019 - link
Oh the irony! All they needed was 20-30 minutes (preferably more) of battery backup for the factory and the irony is that Toshiba makes grid scale battery systems! https://www.scib.jp/en/applications/energy.htm