BSOD when running OA and Visual Studio 2012 Ultimate

[ruirib 2]

Every single time I try to run Visual Studio 2012 with Online Armor, I end up getting a BSOD. This happens both on Windows 7 Ultimate and Windows 8 Pro. If I shut down OA, I can run VS 2012 fine, either editing or building or running apps.

 

This has been going on for since OA 6.0 was released and I have dealt with it simply by shutting down OA whenever I am using VS. Yesterday, however, I was not beyond a hardware firewall when I used it and I realized my patience with this had run out. This needs to be fixed.

 

Besides OA, my only other security app is EAM. 

 

I have the latest dump file from the crash on Windows 8, which I can upload or send to someone, if needed.

[ruirib 2]

Ok, adding my dump file here.

 

 

[Fabian Wosar 390]

I looked through your posts here as well as in our helpdesk and so far and can't find any previous reports about blue screens in combination with Visual Studio 2012 from you. Please understand that we don't spy on our customers. So we can't know if something is going on your system unless you tell us. Trying to wait for a fix for a bug that has never been reported is pretty much pointless. If we knew about the bug in the first place we wouldn't have released the Online Armor version to begin with.

That being said, we will require at least the mini dump, maybe even the kernel memory dump, that was created during the crash. So I would appreciate it if you could attach the mini dump to your next reply. The file attachment options become available after clicking the "More Reply Options" button.

Thanks!

[Fabian Wosar 390]

You were faster than me when attaching the mini dump. Will take a look at it ASAP .

[Fabian Wosar 390]

Hi ruirib,

So far I have been unsuccessful in reproducing the issue on my Windows 7 x64 and Windows 8 x64 test systems, both fully updated, using Visual Studio 2012 Professional including Update 1 and building/running C++ Win32 applications. Does the BSOD occur only when you build specific projects in specific languages or is it pretty much universal on your system?

[ruirib 2]

I have used it mostly to build ASP.NET MVC web apps. It usually happens when editing files.

 

I can't tell you if this is an issue specific to VS Ultimate, but that's the one I run in both the desktop and laptop. I am running it with the Update 2, released very recently, but this also happened with with Update 1 and with the RTM version.

 

This was my first report of the issue, yeah. I should have said that. I have reported other issues, but not this one.

[Fabian Wosar 390]

I don't thing we have a license for the Ultimate version as we don't need TFS and usually settle for Premium, but I can try with the trial version. I will also try building web apps instead of desktop applications.

[Fabian Wosar 390]

Hello ruirib,

I have spent the past couple of days trying to reproduce the problem you are experiencing on my systems. Unfortunately no matter which combinations of OS, Visual Studio versions and Online Armor I try, I haven't been able to blue screen the system. From the mini dump it seems that the system crashes while processing the configuration manager callbacks. From what I can tell based on the rather limited amount of information included in the mini dump it isn't caused by the Online Armor driver callback, but to completely rule that out I would need a kernel memory dump or a full memory dump. Would it be possible for you to upload one of these dumps to either a hoster of your choice or alternatively to one of our FTP servers?

Thanks again for your patience .

[ruirib 2]

Sorry, I have been very busy. I will try to get you a kernel memory dump.

[ruirib 2]

Ok, here is the kernel dump: http://ruirib.net/kerneldump.zip.

 

It's really easy to cause a crash. Consecutive clicks inside a VS editor window accomplish it in a few seconds.

 

Please let me know once you get the file.

[ruirib 2]

Any news on this?

[Fabian Wosar 390]

Unfortunately not. The crash happens when Visual Studio tries to access a specific registry location on your system. Even if I replicate the registry access exactly it doesn't crash on any of my test systems no matter which operating system I test with. The crash doesn't occur directly within our driver either. The crash instead occurs in the Windows code that is related to the registry callback functions way after processing in the Online Armor driver has finished. I also looked for possible signs of memory corruption caused by Online Armor that could explain why the OS code crashes, but no luck so far.

 

Would it be possible to enable driver verifier on your system? If there really is some issue with memory corruption on your system that would be by far the fastest way to find it. I can send you the necessary instructions on how to setup up driver verifier on your system. No additional installations needed as it is integrated within Windows already.

[ruirib 2]

Yeah, I can try that . You need to be aware this happens in both my systems, Windows 7 x64 and Windows 8 x64. So far I have sent only stuff from my Windows 8 laptop, as I have been keeping OA shutdown on the desktop whenever I use VS.

[ruirib 2]

Ok, I've checked how driver verifier can be turned on. A couple questions:

 

1. Shall I do it with all drivers?

2. What settings do you recommend?

[Fabian Wosar 390]

This should work:

1. Delete any existing settings by starting verifier.exe, selecting "Delete existing settings" and finish the wizard.
2. Start verifier.exe again.
3. "Create custom settings"
4. "Select individual settings from a full list"
5. Select only "Special pool"
6. "Select driver names from a list"
7. Only check oadriver.sys
8. Finish and reboot

Also set the crash dump type to full dump. Explanations on how to do that can be found here. Then replicate the issue. Once driver verifier notices an issue it will immediately blue screen the system and write the dump allowing us to take a look at what is going on in the driver the second the actual corruption takes place, if that is indeed the underlying issue.

 

After you captured the full memory dump you can remove the driver verifier monitoring by simply deleting the settings as outlined in step 1.

[ruirib 2]

It's going to take a bit. 1.6 GB won't upload fast.

[ruirib 2]

Full memory dump: http://ruirib.net/memory.zip

 

 

[Fabian Wosar 390]

Thank you. I downloaded the dump and will be looking into it shortly.

[Fabian Wosar 390]

Hi ruirib,

 

Can you please export the following registry key including all its sub keys on your system and send it via mail to [email protected]?

HKEY_USERS\S-1-5-21-3463171804-1537994893-3906066650-1001\SOFTWARE\MICROSOFT\VISUALSTUDIO\11.0\TOOLBOXCONTROLSINSTALLERCACHE\SYSTEM.WINDOWS.CONTROLS.DOMAINSERVICES, VERSION=2.0.5.0, CULTURE=NEUTRAL, PUBLICKEYTOKEN=31BF3856AD364E35

Thanks .

[ruirib 2]

Ok, I have emailed it. FWIW, accessing the key with regedit, while OA was running, caused a BSOD. I had to close OA to be able to access it without issues. The export file was created even with the BSOD. Not sure what were the contents, though, as I accessed it only after a second export. Anyway, it's done.

 

Thank you.

[ruirib 2]

Just an update to this. Adding Visual Studio to the exclusions list stops the BSODs. I should have done it earlier  .

[Fabian Wosar 390]

We have also fixed the issue internally but due to the kind of changes we made we need to test it a bit more extensively than we usually would as many subsystems are affected by the change. A new beta build should be available in the coming weeks though.