 |
Problem Status: RESOLVED Affected Systems: Email and Internet Connectivity Problem Begin: October 18, 2006 Problem Resolved: October 23, 2006 At approximately 4:00 p.m. on Wednesday, October 18th, several computer servers in the PVAMU data center were infected by a new and yet to be named virus. The virus self replicates and attacks server software structures in the system replication processes that make it impossible for a user to log on to the PVAMU email system. It also generates random large data bursts on the network resulting in network routing problems, slowdowns, and outages.
IT Services is currently working with Microsoft and McAfee to mitigate the virus, and while we have made progress in the last 30 hours, we still do not have the problem solved. Unfortunately, the virus destroys vital software resources when it launches causing us to rebuild those resources on each occurrence. We ask your patience as we work towards a resolution to this problem and we apologize for any inconvenience experienced during this outage.
IT Services will update this page regularly to keep you informed. Again, thank you for you patience and understanding in this matter!
Status Update: Thursday October 19, 2006 5:00 p.m.
IT Services has successfully isolated the virus and has manually removed it from all impacted servers in the data center. The offending code has been sent to McAfee for analysis and resolution.
The removal of the virus and the rebuilding of the replication system software resources with Microsoft have allowed IT Services to bring up email for general use. Should you have any difficulty in accessing email services please contact the Help Desk at 2525 or 1-877-241-1752.
Status Update: Friday October 20, 2006 5:00 p.m.
As we continue our efforts to stem the tide of this virus, we have learned that we are not alone in fighting this menace. We have received information from the vendors we are working with that this threat is on the move and wreaking havoc in its wake. Efforts by antivirus vendors and Microsoft have been stepped up significantly and the hope is that a complete remedy for this situation is not far off.
As for our situation, by Thursday evening we successfully brought our email system on line for general use. With the assistance of the Microsoft support team we were able to steel our system software resources against reinfection. However, we did suffer an unintended side effect in that we so well protected our resources that we locked out third party software friendlies which work hand in hand with our Exchange email system.
The system most affected was the mechanism responsible for the delivery of voice mail. You may have noticed that you did not receive any voice mail deliveries via email and that if you tried to leave a voice mail for someone you got a fast busy. This was an unintended consequence of our remedy and we apologize for any inconvenience it may have caused. Again, with Microsoft's help, the situation has been resolved.
We have also learned that this virus significantly impacts pc desktops that do not have the latest Microsoft operating system patches applied. Some of you may notice network connectivity slowdowns our outages sporadically. We have learned that these outages are directly attributed to our virus which uses non patched desktops to wreak havoc with dynamic network routes. IT Services has already begun the task of ensuring that all PVAMU desktops are properly patched. We will use the LAN Desk agent installed and running on your desktop to assist us with applying the necessary patches. It is imperative that we patch all machines using the network to keep them from being used as a repository and replicator for the virus until the antivirus vendors create a solution. If your department operates a lab or you support desktops that do not have LAN Desk installed please contact Mr. T. D. Jefferson, IT Services User Services and Desktop Support director at (936) 261-9300 or tdjefferson@pvamu.edu for further instructions.
Again, IT Services appreciates and would like to thank you for your understanding and patience as we work to eradicate this nuisance and bring our services back to full functionality.
Status Update: Monday October 23, 2006 5:00 p.m.
IT Services worked through the weekend with Microsoft and McAfee support services successfully applying the latest Microsoft patches to on campus pcs, identifying, isolating, and eradicating our virus, and restoring the PVAMU networking environment to normal working order.
McAfee identified our virus as a network worm, a variant of the W32/Sdbot.worm family, with backdoor functionality for the Windows platform. This worm, as does every worm which affects the windows operating system, was self-replicating and exploited vulnerabilities in the windows code.
As Microsoft is extremely vigilant in patching their vulnerabilities, and McAfee is as vigilant in supplying us with antivirus solutions, it becomes imperative that we apply the supplied patches from both vendors to our desktop machines as soon as possible. To that end, IT Services has in place automated processes to download and apply the latest Microsoft patches and McAfee antivirus updates to all systems logging on to the network in the PV domain or any system on the network running LAN Desk. Should you have any question about your system residing in the PV domain or running LAN Desk, please contact Mr. T. D. Jefferson, IT Services User Services and Desktop Support director at (936) 261-9300 or tdjefferson@pvamu.edu for additional information.
In summary, IT Services believes that the virus which attacked the PV network has been contained, removed from our systems, and network services and connectivity have been restored to normal working order. Should you experience any problems or are unable to use email or access the Internet, please contact the Help Desk at 2525 or 1-877-241-1752 at your earliest convenience.
Thanks again for your understanding, patience and cooperation. |