Creaking Patch Tuesday's Viability Rests with Quality, Speed

Today is Patch Tuesday, the 11-year-old procession of security bulletins from Microsoft streamed out automatically to consumers of Windows Update, and pulled en masse by enterprise admins worldwide needing to test each for compatibility.

This is how it’s been done since shortly after Bill Gates’ Trustworthy Computing memo in 2002 set Microsoft on its course of secure software development. But in 2015, as the concept approaches adolescence, are we asking the right questions about the viability of a scheduled patch delivery?

Sure enterprises may be engrained in this rote consumption of security fixes on the second Tuesday of every month, but given that Microsoft is in the middle of a personality overhaul under new CEO Satya Nadella with a vigorous focus on the cloud, and the company’s vaunted Trustworthy Computing group disbanded as a single entity and migrated into several business units inside Microsoft, Patch Tuesday may showing some signs of cracking.

Outside forces aren’t helping much. Zero days dominate the headlines, but affect relatively few until attacks find their way into exploit kits, turning specialized hacks into commodity danger. Google’s Project Zero is the most recent conspirator undermining the value of regular patching cycles; the research team has put vendors on notice that a 90-day countdown starts the second a vulnerability is reported to Microsoft—or any vendor for that matter. And once the 90 days are up, disclosure is full and angst is high.

Patch Quality in Crosshairs

Internally, since TWC in September was integrated into Microsoft’s cloud and enterprise group—coinciding with more than 2,100 layoffs, including several key security people—eyebrows have also been raised about patch quality and timeliness. Most notably, a critical vulnerability in Microsoft’s sChannel, the SSL/TLS implementation in Windows, was patched in November but within days, the patch was pulled back because of issues with TLS negotiations. It was re-issued in short order, but coincidently or not, the situation did not endear anyone to the reorg going on in Redmond.

Even going into today’s Patch Tuesday release, a critical cross-site scripting vulnerability in Internet Explorer affecting Windows 7 and 8.1 users that last week was made public along with proof-of-concept code, still is unpatched and Microsoft has been silent on when a fix is coming. That silence, could in part, be due to the fact that the company recently discontinued providing users with advanced notification of patches, making them available only to premier support customers. Perhaps, security will stop being a marketing differentiator for Microsoft.

“They’re not going to get rid of security, but like Apple, put it more behind the scenes,” said Marc Maiffret, a longtime Windows bug-hunter and current CTO at BeyondTrust. “It’s not going to be the thing they talk about most. It distracts from them being a software and technology company.”

Microsoft’s QA testing of patches is extensive and reportedly separate from the Microsoft Security Resource Center (MSRC) and TWC, which focuses on security research, threat modeling and risk management. Updates are tested against a variety of application and operating system environments for compatibility issues and must meet strict deadlines to be included in a timely fashion to Windows Update. Patches are also tested against third party applications, and Microsoft will insist that patch quality issues have little to do with TWC changes and more to do with advanced and changing threats.

“Microsoft carefully reviews and tests each security update to ensure its quality and that it has been thoroughly evaluated for application compatibility. There are many factors that can impact the length of testing,” said Chris Betz of the MSRC in a statement provided to Threatpost. “Once the update is built, it must be tested with the different operating systems and applications it affects, then localized for the different markets and languages around the world. In some instances, multiple vendors are affected by the same or similar issues, which requires a coordinated release.”

Microsoft’s focus on delivering a consistent schedule of patches helps users inside the enterprise and smaller organizations line up their deck chairs, do compatibility testing and control patch rollouts. These processes are finely tuned compared to a decade ago, and most organizations would not trade Patch Tuesday, say for automatic silent patching, a la Google’s updates to Chrome, for example, experts said.

“The bigger factor that surrounds things like Patch Tuesday is that threats have changed,” Maiffret said. “Organizations like governments or anyone who is a high-value target, has a good chance of getting hit with a zero day, which Patch Tuesday has no bearing on, at least up front. That’s a big part of it: security moving away from the value of one individual vulnerability.”

Automatic Patching Has Its Place

Microsoft, for its part, has not been stagnant with patching. New services such as myBulletins and a revamped Exploitability Index help customers make deployment decisions, while its partner programs such as Microsoft Active Protections Program give participating enterprises and vendors a head’s up on vulnerability details in order to coordinate patch delivery with interdependent products.

“Each customer is unique with varying needs based on their technology environments. With the evolution of cloud computing, more and more customers are taking advantage of the real time updates we provide,” said Betz. “Customers are also increasingly taking advantage of Microsoft Update to automatically provide updates.”

Attackers, however, have the luxury of being able to focus on one bug, but defenders have to look at the biggest risks to their respective environments, hoping they make the right assessments and prioritizations. And this goes well beyond Microsoft to third-party applications such as Flash, Java and others that run everywhere and have been providing attackers with much more tempting targets of late. Yet with the world primarily still running on Windows, especially in smaller organizations, patch quality still gives people pause with regard to going to an automated process.

“I think people would like to be in automatic mode. There’s a huge value to set-it-and-forget-it, but there’s still a risk involved and it’s difficult for people to consume that risk not knowing what could happen,” said Andrew Storms, vice president of security services at New Context, and former security executive at CloudPassage and nCircle. “Large enterprises are always slower moving to the adoption of new concepts and risk, especially with IT. The argument for the other side is what if I could cut a third of my patching costs if I don’t have to patch all the time; if I were a CIO, I would be drooling.”

That, of course, depends on patches that are good to go out of the box, so to speak.

“Any business at the scale of Google or Microsoft have so many complexities that there are going to be unforeseen interactions,” said Tripwire security researcher Craig Young. “That’s why enterprises test patches in a controlled environment to make sure they don’t breach critical business applications before rolling them out to systems. That works. The Chrome model is probably not appropriate if you’re a hospital where all your terminals need a web app interface with insurance providers and if Microsoft updates IE and the web app no longer renders properly, how would you address that situation?”

Environment to Dictate Patching Styles

Katie Moussouris, a former lead security strategist at Microsoft and current chief policy officer at HackerOne, was deeply involved in the development of Microsoft’s coordinated disclosure program and developing strong relationships with vulnerability researchers and brokers. She says vendors need to sharpen patch development where quality and speed go hand in hand. This takes on more relevance with the so-called Internet of Things, where embedded computers often don’t have simple patching mechanisms yet play critical roles in manufacturing, health care and personal environments.

“Patching style is something that definitely has to evolve as what makes up the bulk of internet traffic starts changing,” Moussouris said. “Mobile devices are difficult to patch, and are not patched on anyone’s schedule. Many are not designed to be patched either; they’re designed to be upgraded or thrown away in two years.”

Microsoft, meanwhile, has taken steps to make exploitation more difficult for attackers. The introduction of memory corruption mitigations such as ASLR and DEP into Windows and Internet Explorer have made buffer overflow vulnerabilities less of a hassle than a decade ago. Free tools such as the Enhanced Mitigation Experience Toolkit (EMET) are often a stopgap for zero-day vulnerabilities until Microsoft can release a scheduled or out-of-band security bulletin.

“Microsoft has focused on a higher level of mitigations, knowing how high to raise the bar to make exploitation really hard,” Maiffret said. “I hope they keep their eye on mitigations, not just EMET but also the underlying operating system.”

For the time being, Microsoft won’t retire Patch Tuesday and its high-paying enterprise customers likely won’t let them. And in the end, Patch Tuesday is still relevant on many fronts, and the processes are still superior to many third-party patching processes.

“Stepping back, you have to ask: ‘What’s the relevance of Microsoft vulnerabilities in attacks and exploits?'” Maiffret said. “Microsoft software is still relevant and part of targeted attacks; you still see IE targeted attacks happening, but at the same time, you’re seeing an increase of third-party apps in targeted attacks. That’s the biggest shift. Microsoft is slightly putting security in the back seat, not doing less internally, but in visibility. That mirrors what’s happening from the attackers’ perspective; it’s just as important to find a Flash or Java vulnerability versus a Microsoft vulnerability.”