reerden

Massive IO issues with 12.1.1.7014

Recommended Posts

I'm having issues with applications being unable to access any files after running the on demand scan a few times. A reboot only fixes this issue temporarily.

The event viewer is full of errors indicating apps can't access files because they are already in use. This causes gigantic issues like settings disappearing and even being unable to fully boot after login. A quick inspection with Sysinternals Process explorer reveals that the culprit is the a2service, which keeps open the file handlers after a manual scan.

I've put my update feed in delayed and my EAM is now back at 12.0.1.6859. All issues have disappeared.

This is on a Windows 10 professional 64 bit. The only other security software I have installed is hitmanpro alert.

I'm unable to provide you with a debug log as EAM is unable to save any logs when it happens, since the a2service keeps locking the log files.

 

EDIT: problem is also present on another PC. I tried removing hitmanpro to see if it caused the issue, but it is still present.

Share this post


Link to post
Share on other sites

Hi reerden,

Looks like I have seen this problem before with some customers sinds updating to 7014. The systems are also Windows 10. 

I was unable to reproduce the problem on my systems with Windows 10. I don't have advanced debug logs.

Did you also experienced that Start menu didn't open?

 

I'm sure Emsisoft support will help you as soon as possible. 

Share this post


Link to post
Share on other sites
2 hours ago, PC reset said:

Hi reerden,

Looks like I have seen this problem before with some customers sinds updating to 7014. The systems are also Windows 10. 

I was unable to reproduce the problem on my systems with Windows 10. I don't have advanced debug logs.

Did you also experienced that Start menu didn't open?

 

I'm sure Emsisoft support will help you as soon as possible. 

Yes. Start menu locks up too.

EDIT: I'm able to reproduce the problem by running the malware scan a few times. Usually, the first indication the problem starts is that the event viewer shows that windows can't write live tiles to its cache. The more often I run the scan, the worse the problem becomes.

I can see the EAM service creating file handlers for files it's scanning using process explorer. They're never released and keep building up until the system becomes unusable.

 

EDIT2: I also have bitlocker active on both machines. Maybe that has something to do with it.

Share this post


Link to post
Share on other sites
Quote

EDIT2: I also have bitlocker active on both machines. Maybe that has something to do with it.

Maybe, but the systems I have seen didn't have Bitlocker enabled on their system.

Do you have the latest updates of Windows installed? Build 1607?

 

Share this post


Link to post
Share on other sites
13 hours ago, PC reset said:

I was unable to reproduce the problem on my systems with Windows 10.

Do you have an SSD? If this is related to the known I/O issues our scan engine is having, then it would only manifest itself on a magnetic hard drive.

 

13 hours ago, reerden said:

I'm able to reproduce the problem by running the malware scan a few times. Usually, the first indication the problem starts is that the event viewer shows that windows can't write live tiles to its cache. The more often I run the scan, the worse the problem becomes.

I assume you are running complete scans, rather than stopping them and restarting them?

 

13 hours ago, reerden said:

I can see the EAM service creating file handlers for files it's scanning using process explorer. They're never released and keep building up until the system becomes unusable.

I'll ask our developers about that, as this doesn't sound like the I/O problems I am already familiar with.

Share this post


Link to post
Share on other sites

I just spoke to one of our developers about this, and they would like some debug logs. Here's how to get the Scan Engine Debug Logs they asked for:

  1. Download and open the ZIP archive at this link.
  2. When the Emsisoft_Debug_Tool ZIP archive opens, double-click on the batch file called Emsisoft_Debug_Tool that was inside.
  3. If Windows asks you if you want to allow the Windows Command Processor to make changes to your computer, then say Yes.
  4. When you see a bright blue screen with a menu, click on it to make sure it is in focus, and then press 3 on your keyboard followed by Enter.
  5. Press Enter again to return the the main menu, and then Enter again to close the blue window.
  6. Restart your computer (on Windows 10 and 8.1 please right-click on the Start button, go to Shut down or sign out, and select Restart in order to do a full restart).
  7. Open Emsisoft Anti-Malware, and run your scan again to reproduce the issue.
  8. Once the issue has been reproduced, close Emsisoft Anti-Malware.
  9. There should be a new file named ScanEngineDebug.log in the Emsisoft Anti-Malware folder (usually C:\Program Files\Emsisoft Anti-Malware or C:\Program Files (x86)\Emsisoft Anti-Malware).
  10. Once you find the ScanEngineDebug.log file, please attach it to a reply.

After sending us the ScanEngineDebug.log file please be sure to run the Emsisoft_Debug_Tool (steps 1-6) again to turn off Scan Engine logs (option 4 in the menu this time).

Note: On 64-bit editions of Windows the Emsisoft_Debug_Tool will have extra options for the 32-bit (x86) version of Emsisoft Emergency Kit. Only use these if you were specifically having trouble with the version of Emsisoft Emergency Kit that was located in the bin32 folder.

Share this post


Link to post
Share on other sites

I'll send a debug log as soon as I get home.


I'm also in contact with Mr Biggar through the helpdesk mail. I've send him a FRST log.

Both systems have an SSD. No hard drives.

And yes. These are complete scans. The problem becomes worse the more scans I run without a full reboot.

Share this post


Link to post
Share on other sites
Quote

Do you have an SSD? If this is related to the known I/O issues our scan engine is having, then it would only manifest itself on a magnetic hard drive.

Yes I use SSD on all systems :rolleyes:

Share this post


Link to post
Share on other sites
6 hours ago, GT500 said:

I just spoke to one of our developers about this, and they would like some debug logs. Here's how to get the Scan Engine Debug Logs they asked for:

  1. Download and open the ZIP archive at this link.
  2. When the Emsisoft_Debug_Tool ZIP archive opens, double-click on the batch file called Emsisoft_Debug_Tool that was inside.
  3. If Windows asks you if you want to allow the Windows Command Processor to make changes to your computer, then say Yes.
  4. When you see a bright blue screen with a menu, click on it to make sure it is in focus, and then press 3 on your keyboard followed by Enter.
  5. Press Enter again to return the the main menu, and then Enter again to close the blue window.
  6. Restart your computer (on Windows 10 and 8.1 please right-click on the Start button, go to Shut down or sign out, and select Restart in order to do a full restart).
  7. Open Emsisoft Anti-Malware, and run your scan again to reproduce the issue.
  8. Once the issue has been reproduced, close Emsisoft Anti-Malware.
  9. There should be a new file named ScanEngineDebug.log in the Emsisoft Anti-Malware folder (usually C:\Program Files\Emsisoft Anti-Malware or C:\Program Files (x86)\Emsisoft Anti-Malware).
  10. Once you find the ScanEngineDebug.log file, please attach it to a reply.

 

After sending us the ScanEngineDebug.log file please be sure to run the Emsisoft_Debug_Tool (steps 1-6) again to turn off Scan Engine logs (option 4 in the menu this time).

Note: On 64-bit editions of Windows the Emsisoft_Debug_Tool will have extra options for the 32-bit (x86) version of Emsisoft Emergency Kit. Only use these if you were specifically having trouble with the version of Emsisoft Emergency Kit that was located in the bin32 folder.

Here's the scan log. Did a few scans and had the problem somewhat less severe. Rebooted and immediately ran a scan and had the problem again. All scans were fully completed.

tmp_8667-ScanEngineDebug14045624.log

Share this post


Link to post
Share on other sites

We believe this issue has been fixed in a beta update. If you would like to try it, then here's what to do:

  1. Open Emsisoft Anti-Malware.
  2. Click on Settings in the menu at the top.
  3. Click on Updates in the menu at the top.
  4. On the left, under Update Settings, click on the box to the right of Update feed and select Beta from the list.
  5. Click on the Update now button on the right side.

Share this post


Link to post
Share on other sites
On 12/23/2016 at 6:49 AM, GT500 said:

We believe this issue has been fixed in a beta update. If you would like to try it, then here's what to do:

 

  1. Open Emsisoft Anti-Malware.
  2. Click on Settings in the menu at the top.
  3. Click on Updates in the menu at the top.
  4. On the left, under Update Settings, click on the box to the right of Update feed and select Beta from the list.
  5. Click on the Update now button on the right side.

 

I'll check as soon as I get the chance. Thanks for help and the quick response.

Share this post


Link to post
Share on other sites

I'm curious: how does a problem like this get out into the wild in the first place?     If doing a few on-demand scans uses up file handles, surely that's the sort of thing that automated testing of new builds would, or should, pick up.   So, does that mean you don't do automated testing?   And, if you do, when you solve a problem do you also modify the test framework to try to detect that sort of thing happening in future builds?

Share this post


Link to post
Share on other sites

The handles aren't leaked. They get freed up just fine. We just open up a lot of them at the same time in the first place, which on some systems causes issues. For the vast majority of our users, it doesn't cause any issues, and neither does it on any of our test systems. 

Share this post


Link to post
Share on other sites
20 minutes ago, Fabian Wosar said:

The handles aren't leaked. They get freed up just fine. We just open up a lot of them at the same time in the first place, which on some systems causes issues. For the vast majority of our users, it doesn't cause any issues, and neither does it on any of our test systems. 

That's weird because they weren't freed on my systems, even after 30 minutes or so. I had problems with the entire system hanging and applications losing their settings profile because they couldn't access it. Caused by a read error caused by file locking. The file locking was caused by the a2service according to process explorer. Removing or downgrading EAM also solved the issue.

I'm also not sure what is so different about my systems, other than hitmanpro alert being installed (which I also removed for testing). I only have a few applications installed, and even less on the other PC.

Share this post


Link to post
Share on other sites

As I said, opening that many handles at once causes issues on some systems. After that happens, the system is in an undefined state. That being said, the problem is not that the handles aren't freed, because they are, provided the system doesn't start acting all weird because too many handles have been opened. In either case, we switched it so we at most will open 2x the thread count of files during a scan task.

Share this post


Link to post
Share on other sites

Info: I had similar problems. But I think ... this problem is similar to this: "Internal Processing Error".

After "own Scan C:" I get the Windows-info "no access to file ..."; Also SpeedCommander could not access its own configuration files in folder %userprofile%\AppData\Roaming\SpeedProject... also universal apps could not start. All without EAM-error message.

An EAM-error message "Internal Processing Error" at "scheduled scan" came after this scan, after an signature update; with the same problems as mentioned above.

After reboot and changing to EAM-beta, I've no problems with EAM-beta, the current beta is stable here.

Share this post


Link to post
Share on other sites
2 hours ago, Solution-Design said:

After reboot and changing to EAM-beta, I've no problems with EAM-beta, the current beta is stable here.

That's great. Thanks for letting us know. ;)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.