Update your shit, yo.

Discussion in 'General Discussion' started by HittingSmoke, Jan 4, 2018.

Update your shit, yo.

Discussion in 'General Discussion' started by HittingSmoke, Jan 4, 2018.

  1. HittingSmoke

    HittingSmoke AoA Emeritus



    tl;dr: Two critical hardware-level vulnerabilities were found in pretty much every CPU on the market. The more serious vulnerability affects nearly every Intel CPU sold since 1995. The fixes involve patches to everything from the kernel to the compiler to the userland application. Update your shit. All of it. Your desktop OS, your servers, your phone, your browser. Today Mozilla confirmed they were able to exploit Spectre via javascript and that a patch was issued to mitigate it. Keep an eye out for updates to pretty much every piece of software you use in the coming days.

    No so tl;dr:

    Meltdown is the codename for a vulnerability discovered in every Intel CPU that supports out of order execution. Unless you're on an Atom from pre-2013 or an Itanium, this means you if you're on an Intel CPU. It allows reading of privileged kernel memory space from unprivileged userland applications. It is easily fixed with kernel page isolation implemented in the kernel, but this comes at a performance hit. The performance impact ranges drastically from 5% to as high as over 30% depending on the specific workload. For gaming you're on the low end. If you run a server with a lot of parallel code execution going on like a virtual host you're going to see a much, much greater performance hit. Unfortunately, Intel's response to this has been absolutely abysmal. They went immediately into PR mode and started deflecting instead of doing what every tech company should do when a major vulnerability is discovered, keep to the security facts and do everything you can to mitigate the problem. This is very seriously going to impact the decision I make in my next CPU purchase. I was already looking at EPYC chips for my next server build because of the high thread count per dollar and this issue has pretty much settled that discussion.

    Spectre is the codename for the less serious of the two issues, which is a vulnerability in every single CPU that supports speculative execution. That's pretty much every x86 and ARM processor on planet Earth, so you don't need to bother checking your CPU model. Your phone and your router is vulnerable as well. Hell your TV likely has an ARM processor in it. Luckily it's much more difficult to exploit but also more difficult to patch. So far AMD's response has been to-the-facts disclosures and education.

    Mozilla has demonstrated that both Meltdown and Spectre can be exploited via javascript in preliminary investigations. Firefox has been patched with mitigation techniques but it's going to be a while before these holes are properly plugged and it's going to be an ugly few weeks as new information comes out. Chrome is ironically (since Google disclosed by exploits) holding off a patch until the Chrome 64 release at the end of January. Until then Google recommends enabling the experimental Strict Site Isolation feature in Chrome 63. Though from my reading of the exploits I don't see how this does fuck-all since it's a kernel-space issue.

    Every major Linux distro should have a kernel update pushed by now and Microsoft released a patch yesterday that will go out next Tuesday for everyone. The January security update for Android supposedly has some patches related to the exploits but is waiting on a patched kernel still. I don't know what the hell Apple is doing. I've not heard anything from them.

    So uhh, yeah. Update your shit.
    Cirsphe, Fell and syberghost like this.
  2. syberghost

    syberghost Guild Admin Staff Member

    This is going to be a lot of text, so let me preface it by saying breathe, we've had things this bad before and they haven't destroyed the entire concept of computers, it's going to be OK. But it's also bad.

    More importantly than updating this instant, turn your automatic updates on if you've been leaving them off. There are a series of updates that have to happen soon, and will be more. These two bugs will be with us for years.

    As an example of the failcascade going on right now:

    Many antivirus vendors were using undocumented and unsupported kernel hooks. The Meltdown fix in Windows breaks them, in a way that causes a bluescreen. So Microsoft is requiring a registry entry that tells it "my AV is compatible with this patch" before the patch will load. Many of the common AV vendors don't have their fixes ready yet; some are dragging their feet, some are working as fast as they can but require major changes that have to be very thoroughly tested first.

    So depending on your AV vendor, if you install patches right now, you won't get this fix!

    Microsoft Security Essentials (built into Windows 10, free for Windows 7) is of course already good for the patch, but isn't the best AV choice (it's in the top tier, just not the best), so you can safely fall back to it if you like; but once you've falling back and installed the patch, you cannot then reinstall your favorite AV because it might crash your system until they fix it.

    It's a fucking mess right now, but if you turn on all the automatic updates, it will fix itself in the near term for Meltdown.

    Spectre is in some ways worse; it's not just that you can patch for it soon, the patches are going to be bandaids against specific exploits. The real fix is going to involve installing a new CPU *THAT HASN'T EVEN BEEN DESIGNED YET, MUCH LESS BUILT*, so you're going to be patching for a while, as HS said. We expect this one to be with us for a decade or more.

    TURN UPDATES ON IF YOU TURNED THEM OFF. Gamers, especially streamers, are bad about this. Turn 'em on.
    --- Double Post Merged, Jan 4, 2018, Original Post Date: Jan 4, 2018 ---
    More on the failcascade:

    --- Double Post Merged, Jan 4, 2018 ---
    The spreadsheet he's maintaining for tracking AV vendors. Mine's on the naughty list.

    Fell and HittingSmoke like this.
  3. syberghost

    syberghost Guild Admin Staff Member

    A thing that I am not yet sure is a thing, but think is a thing:

    A first everybody was benchmarking server stuff, and we were digesting the fact that stuff that takes in a lot of transactions and shoves them into databases is going to be a lot slower, so we have to account for that in our capacity planning this year (thank GOD my company's Peak ended like literally yesterday and I've got 10 months to fuck with this), and stuff that mostly sits in user space and doesn't move around between parts of the kernel won't be affected much. So of course we said "gaming is probably OK, unless the GPUs are affected. Oh, shit, are the GPUs affected?"

    And then the GPU folks were like "oh fuck, uhm, shit, let me read this white paper, and benchmark some... nah, fam, we're good."

    So then just to prove it everybody has been benchmarking games, and yeah, games are good, you're all fine. Except we forgot something:

    Most games now are MMOs. The back end takes in shitloads of data from the interwebz, does some user space shit to it, then puts it into a database, and vice versa. That's bloop through the kernel, bloop through userspace, bloop back to the kernel FUCK. MMOs just hit the worst case backend performance.

    I find it difficult to think that every online game is going to absorb a 12-17% increase in their server costs without having to pass that on to the gamers.
    HittingSmoke likes this.
  4. HittingSmoke

    HittingSmoke AoA Emeritus

    Oh shit. Yeah. And a lot of MMOs still haven't made it to easily scalable "cloud" hosting yet so it's not just a click of a button to ramp things up.
  5. Arcfire

    Arcfire Guild Admin Staff Member

    RIP and syberghost like this.
  6. Fell

    Fell Guild Admin Staff Member

    i was just reading about this on CNN, not the greatest haven for tech knowledge and figured i'd rush over here to ask if anyone had informed opinions. I love this group and yes i'm on the always update side of the house. That said i'll want to think about my AV response a bit more carefully. You guys absolutely rock, i didn't even have to read anything twice to digest it...
    HittingSmoke likes this.
  7. syberghost

    syberghost Guild Admin Staff Member

  8. HittingSmoke

    HittingSmoke AoA Emeritus

    Since it's my job I'll probably be digging into the AV side of things this week, though I'm technically on vacation until Thursday. My always-current recommendation is to keep a MBAM subscription. It's the best piece of software made for end user security and unlike most AV it's worth the money.
  9. syberghost

    syberghost Guild Admin Staff Member

    MBAM is all good with the patch and the registry key.
    HittingSmoke likes this.
  10. syberghost

    syberghost Guild Admin Staff Member

    So, Epic games had problems with their Fortnite login servers. The root cause:

    Meltdown patch tripled the CPU utilization.
    HittingSmoke likes this.
  11. HittingSmoke

    HittingSmoke AoA Emeritus


    syberghost likes this.
  12. syberghost

    syberghost Guild Admin Staff Member

  13. Fell

    Fell Guild Admin Staff Member

  14. HittingSmoke

    HittingSmoke AoA Emeritus

    Note that the Meltdown patch from Microsoft is rendering some AMD machine unbootable. It shouldn't roll out to any more affected machines but if you have a machine that's stuck at the Windows boot splash recently and it's got an AMD CPU, that's likely the culprit. You can uninstall the update offline using DISM from a bootable Windows installer.
  15. syberghost

    syberghost Guild Admin Staff Member

    Microsoft has temporarily halted the patch to systems with those processors, so if it hasn't happened yet you're OK.

    Evidently it came as a shock to them that their system requirements for Windows 10 includes 13 year old processors.
    --- Double Post Merged, Jan 9, 2018, Original Post Date: Jan 9, 2018 ---
    Also; it turns out that NVIDIA GPUs are subject to some of this, and now there are patches. The patch notes in the GeForce Experience app don't mention Spectre, but that's part of what they're for.
    Fell likes this.
  16. HittingSmoke

    HittingSmoke AoA Emeritus

    Intel solidified my position, which is quite a feat considering how long I've been 100% Intel. All of my CPU purchases for the foreseeable future will be AMD.

    Intel has decided that their patch is a "feature" to be enabled manually by the user. Intel has made it very clear security is not a priority. That's not acceptable.
  17. syberghost

    syberghost Guild Admin Staff Member

    Meanwhile, all the OS vendors are rejecting the kernel patches Intel contributed. Linus Torvalds called them "absolute garbage" and asked if they were "fucking insane", and that was all BEFORE he stopped being nice about it.

    The Linux and VMWare patches cause random reboots.
    HittingSmoke likes this.
  18. syberghost

    syberghost Guild Admin Staff Member

  19. Cirsphe

    Cirsphe Veteran Member

    Just note that the Apple response to this was also pathetic. One of the last vendors to get the patch out.

    Also, I thought the MS patch would be enough to protect against spectre and meltdown, there is a Intel patch as well that is required? I read about it and was unsure how it fits into the grand scheme of things.

    Btw, this all dropped while I was on paternity leave AND the day everyone in Japan came back from new Year holidays. If it dropped the day before, it would have been so much easier to deal with this.
    AndyCapp likes this.
  20. syberghost

    syberghost Guild Admin Staff Member

    There are going to be patches for this periodically for at least a decade.
    AndyCapp likes this.

Share This Page