possible virtual memory problem with a2service.exe

[JeremyNicoll 58]

Using EIS v11.6.1.6315 on a 64-bit W8.1 system.

I noticed a few days ago that the 'Private Bytes' memory figure (as reported
by Process Hacker), or 'Commit Charge' (task manager) grows and grows for
a2service.exe.

I realise a lot of this figure is virtual storage, but there's a finite pool of that

available.  (Please don't give me a lecture on paging - I've actually written a

paging subsystem in the past.)  OTOH, I don't fully understand the impact of

this figure in Windows OSes.

You may recall I said a few days ago I was going to try running EIS with debug
logging on all the time; when I noticed this creeping increase in Private Bytes
I turned logging off and did a complete power-off shutdown, and rebooted.

Since then (just after midnight, very early on Friday):

- immediately on login to Windows, a2service showed 'Private Bytes' was 472 MB.

- at FRI 00:22    487 MB
- at FRI 07:15    517 MB
- at FRI 09:31    542 MB     (then the machine 'slept' for around 12 hours)
- at FRI 21:54    687 MB
- at SAT 11:00    817 MB
- at SAT 20:20    1.3 GB

If this growth continues at this rate, there's going to be a problem in a week
or two.

[JeremyNicoll 58]

          SUN 0806      1.43 GB
          SUN 2011      1.74 GB
              (machine then slept all night)                             
          MON 1111     1.73 GB
          MON 1700    1.92 GB              
          TUE 0008     1.99 GB
 

[JeremyNicoll 58]

See task manager screenshots

Download Image

Download Image

[GT500 645]

Private Bytes is essentially the amount of memory reserved for the process to use, and the Working Set is what is actually being used.

Let me run this by a developer, and see if they want some debug information.

[JeremyNicoll 58]

What I would expect is that when a program requests allocation of some (virtual) memory
the OS would when it satisfies that make sure that it was capable of 'backing' it with
either real RAM or a slot in the paging file.  After all, when your program places data
in that memory it has to be stored somewhere.  I would not expect the OS to allocate
more pages than it could back, so on my machine with 8 GB RAM and a 4.47 GB paging file,
I expect (apart from a small amount of memory used by one of the graphics cards) there
to be a maximum of about 7.9 + 4.47 = 12.37 GB of memory available.

If an app gets virtual storage from the OS and never writes to it then that page will
never actually get physically swapped to the pagefile, because that would be a waste of
time - it's got nothing in it - but the OS still has to expect that one day it may need
to be saved and there has to be somewhere to put it - either real RAM, or a slot in the
paging file.

I think that the 2 GB 'Private Bytes' size represents all the pages that have been
allocated (or just maybe the sum of all the allocated areas, so the sum of the pages
which contain them would be greater), and that Working Set is the subset of the total
number of allocated pages which are actually in RAM at the moment.

For the OS to have allocated 2 GB of virtual storage to EIS, the paging tables must have
at least 524,288 (ie 512k) entries describing that large number of pages.  That alone is
wasting a certain amount of (I suspect non-paged) pages of RAM, though it will presumably
be attributed to the system rather than EIS.

[JeremyNicoll 58]

I've PMed you a link to a file saved from the SysInternals VMmap program, showing much more detail about the running a2service's virtual memory.

[GT500 645]

It's not abnormal for there to be more memory reserved for an application than it would normally use. I think it has to do with how Windows managed virtual memory, and wanting to make sure that if memory usage suddenly spikes for a process there is memory reserved for it to ensure that it doesn't try to use more memory than is available.

If you make a backup of your settings, and then reset everything back to factory defaults, does this issue still happen?

[JeremyNicoll 58]

> It's not abnormal for there to be more memory reserved for an application than it would
> normally use

Indeed, that's true.  But that's the difference between 'reserved' and 'committed' pages,
and the task manager 'Memory' (as opposed to 'details') screenshot shows the commit charge
was very high.


> I think it has to do with how Windows managed virtual memory, and wanting to make sure
> that if memory usage suddenly spikes for a process there is memory reserved for it to
> ensure that it doesn't try to use more memory than is available.

I've not read anything (and I've read a lot on VM management in the last two days; see
below for URLs for a series of illuminating articles, if you've time to look at them)
that suggests that the OS would reserve vm pages for a process off its own bat.  The
process has to ask for the pages.



I hadn't originally understood every aspect of what I saw in the 'Memory' screenshot;
it showed the non-paged pool using 4.7 GB.  This is vm that's NEVER paged out, ie always
located in physical RAM and explains why (as well as seeing EIS's Private Bytes going up
& up why I was seeing real memory use going up & up).  The NP pool contains the kernel,
OS data structures that must be in RAM so that eg interrupts can be handled, those for
mutexes/semaphores, paging control tables, etc, and storage acquired by drivers (or I
suppose any other kernel-state program that asks for NP storage).

By late last night the size of the NP pool had grown so big (along with EIS's Private Bytes
which was by then 2.35 GB) that I rebooted.  I was scared that the system would crash in a
catastrophic fashion if left running.  I took screenshots (attached) of the memory summary
(from Process Hacker) immediately before and after the reboot (a full 'cold' reboot) and
it is interesting to see that the NP pool on the rebooted system was using only 119 MB, a
huge amount less than 4.7 GB!   Something else I noticed when watching these displays

is that the numbers of NP allocations each second is usually far greated than the number

of frees.  I don't know if that's normal (though I guess it might be typical of a system with an

out-of-control growing NP pool).

EIS's Private Bytes after the reboot was back to the figure of 486 MB, but since then it has
climbed to 553 MB.  I don't know if it will again climb & climb.

Something odd happened just before I rebooted.  I'd signed out of my day-to-day userid & in
as my admin one.  I did this because there was a minor backup I wanted to do from the admin
id (and as I've once a long time ago experienced a windows hang after a sign-out & sign-in
I try not to do that when I'm not prepared to reboot if I have to).  Also I wondered if the
other user would also show high memory use - it did.  Anyway, as soon as that user's desktop
came up (some apps, eg Dropbox, which run on my daily id don't start there, so it's quicker
to start) I got an alert from EIS saying it had just done a software update and needed to
restart its application.  After it had done so I looked at the EIS update logs, because I
was puzzled.  I saw no signs of a software update having just been issued.  It seemed as if
it was a pending action from the software update of a few days ago.  Is it right that a user
session would need an app restart when it has just been logged-in, to activate a software
change that was issued several days ago, which in any case had already been implemented via
my day-to-day userid?  The admin userid had not been 'disconnected' - I never do that - and
in any case the whole system had been rebooted a few days earlier (which is when I started
recording memory use).  Very odd.


> If you make a backup of your settings, and then reset everything back to factory defaults,
> does this issue still happen?

I'm not going to try that yet.  I'd like to see if we can find out what's grabbing the
storage.  As it is, the reboot I felt forced to do last night in the interest of system
(especially FS) integrity, might have lost us the chance to find out, but fortunately
(or not, depending on your point of view) that looks not to be the case.

Also... I've not changed any settings in EIS in the last few days, apart from - now before
two boots ago - having had debug logging on for a while.


Useful URLs:

https://msdn.microsoft.com/en-us/library/windows/hardware/hh439648%28v=vs.85%29.aspx
- a good clear overview of paging, user & system space, page & non-paged parts of latter

https://blogs.technet.microsoft.com/markrussinovich/2008/07/21/pushing-the-limits-of-windows-physical-memory/
- there's a 'meminfo' tool mentioned in this that digs details out of the PFN database
  which records what's in the pages etc, but unfortunately it doesn't work on my system;
  it was after I read about this that I found the SysInternals RAMmap and VMmap utilities.

https://blogs.technet.microsoft.com/markrussinovich/2008/11/17/pushing-the-limits-of-windows-virtual-memory/
- describes the commit limit - that all /committed/ vs must be backed either by ram or paging file.
- describes "Private Bytes" fairly accurately (discussion in the comments shows that even Mark R's
  initial description wasn't quite the whole story)

https://blogs.technet.microsoft.com/markrussinovich/2009/03/10/pushing-the-limits-of-windows-paged-and-nonpaged-pool/
- describes eg the non-paged pool - areas where the OS and device drivers store their data
  essentially everything that must never be paged out (so will be in real RAM)

https://msdn.microsoft.com/en-us/library/aa366778.aspx
- Memory limits for each version of Windows, eg how big can a user address space be?
 

Download Image

Download Image

[JeremyNicoll 58]

Oh... did your developers have any ideas?   This did all start after the last set of software updates for EIS.    Is there

any diagnostic utility they'd like me to run, or - as I don't know my way around such things - anything they'd like me

to install so that they could use them in a remote access session?

[Yilee 1]

Oh... did your developers have any ideas?   This did all start after the last set of software updates for EIS.    Is there

any diagnostic utility they'd like me to run, or - as I don't know my way around such things - anything they'd like me

to install so that they could use them in a remote access session?

 

Hi and thanks for all of your hard work on this issue and others that I have followed. I'm still using the stable update .6315 from April 6, 2016. I stopped updating since the last stable release .6338 on April 18, 2016 to read the Blog Issues so as to avoid bad update patches etc... I prefer to do without signature updates until I am sure the coast is clear. You state that the problem is a result of a recent update. I presume you are referring to .6338 and that the problem would also apply to Emsisoft Anti-Malware (EAM) also??? If so, if the problem does turn out to be a real bug, how bad is it? The issue is not something that I have experience with but I do understand the gist of what you are explaining.

 

So, for the average user not running any VM's with 16 GB of physical memory, no paging file on drive c:(use SSD) but I use a paging file on Drive D: to capture a Complete Memory Dump; How long would you say in Days that it would take for the problem to crash the OS?  I ask because if this is a real problem then very soon other members will soon be blogging about this issue. This is potentially a bad problem if it turns out to be accurate! 

Thanks

[JeremyNicoll 58]

Yilee, thanks for your thanks...

 

When I first reported this I was using EIS v11.6.1.6315, and - actually - I'd not noticed that a newer version had come along.  Maybe that's what took

me by surprise, as described in post #8 above.

 

I'm not sure that I'd want to abandon signature updates...    I don't know if it's possible to keep getting signature updates but not get executable code

updates at the same time.  Do signature updates sometimes need revised executables to work?

 

To be honest I'm surprised that no-one else has reported a problem.  It makes me (andmaybe Emsisoft) wonder why it is that my system has the

problem.  Then again, maybe very very few users monitor resource use on their machines, and even if they do, might not see EIS as part of the

cause, if it is.  There's a possibility that EIS is a victim of a problem in the system, not a cause.  Who can tell?

 

However, for me, when memory use climbs high enough that either task manager or Process Hacker (or the broadly similar Process Explorer tool)

show that the OS is gobbling up real RAM and at the same time EIS is apparently using a vast amount of virtual memory (and that's running out too)

it's reasonable to expect a problem sooner or later.  I don't know enough about W8.1 to know how close to total memory exhaustion one can let a

system get to safely.  If one lets memory use get too high, then there may not be enough to allow the system to do a controlled shutdown (I mean

if I tell it to close), and if it gets even worse, you would eventually expect a BSOD.. and possible file corruption (as with any other BSOD).

 

It may be that last night when I eventually rebooted that I could have left the machine for another few hours.  Maybe another whole day.  But if the

system was going to fall over in a day or so, there was no point in pushing my luck.  I'm sorry I can't provide a more concrete answer.

[JeremyNicoll 58]

I should have added...  I'd been noting memory use figures every few hours since I first noticed a problem.   As I've said above, I use 'Process Hacker' to monitor all

sorts of things in my system, and I have it place a small icon on the taskbar showing physical memory use.  It was the increasing (well past normal) display there

that first lead me to look closely at both virtual & real memory use, and then to see a2service's growing virtual memory problem.    The displays in Process Hacker

of memory use (and many other values) are similar to those one can get in Task Manager (if you right-click its column titles and turn on lots of columns that are

normally hidden.    I'm not sure if task manager can also put tiny graphs onto the systray so you can see a sort of summary of eg cpu use, or whatever.

 

Task manager will however show you a summary of memory use.  The only advice I can really give is that you look to see what values are being shown, and see if

they are growing.  If they are, you'll have to make your own mind up about how nervous you get as they reach maybe 80%, 90%, 95%... of their maximum possible

values.  It's hard for you, if you don't know what your system's normal values are.

[Yilee 1]

I should have added...  I'd been noting memory use figures every few hours since I first noticed a problem.   As I've said above, I use 'Process Hacker' to monitor all

sorts of things in my system, and I have it place a small icon on the taskbar showing physical memory use.  It was the increasing (well past normal) display there

that first lead me to look closely at both virtual & real memory use, and then to see a2service's growing virtual memory problem.    The displays in Process Hacker

of memory use (and many other values) are similar to those one can get in Task Manager (if you right-click its column titles and turn on lots of columns that are

normally hidden.    I'm not sure if task manager can also put tiny graphs onto the systray so you can see a sort of summary of eg cpu use, or whatever.

 

Task manager will however show you a summary of memory use.  The only advice I can really give is that you look to see what values are being shown, and see if

they are growing.  If they are, you'll have to make your own mind up about how nervous you get as they reach maybe 80%, 90%, 95%... of their maximum possible

values.  It's hard for you, if you don't know what your system's normal values are.

 

 

Thanks for the additional info. There are a lot of tangents that I could go off on, but I will try to limit myself. I'm glad that I noticed your topic because in the end I will end up becoming more familiar with proper memory usage. I did what you suggested. I enabled most all the memory columns in task manager and wrote down all the values for a2sevice as well as values for physical and kernel memory values and will monitor for changes over the next few days. I also could use Sysinternals process Explorer if you believe it would be much better for this situation.  My physical memory usage is at 53 to 54%. A2servise uses the most memory out of the running processes at an average of 5,800,200 K for working set, peak working set and private working set with commit at around 6,375,656 K. I have been booted for about 2 whole days. The next process  in line runs at around 165,000 K, which is much lower and then the other remaining processes are much lower. The following is applicable in my case:

 

Win 7 Ultimate 64Bit, Sandboxie with dropped rights for most internet facing programs, External Security Gateway(Zywall USG100) with daily IDP and Antivirus Definition updates and I have all compressed HTTP packets that can't be inspected by the external antivirus destroyed. I am very comfortable without having my EAM signatures updated for a few days. I also block all unused or unassigned IANA protocols at the Security Gateway in all directions. I also for now force only IVp4 and block all forms of IVp6(too hard to design IPv6 firewall rules). I also only use TLS 1.1 and 1.2 and only allow strong ciphers and encryption signatures. The few times that over the last 6 yrs that something got onto my computer were either quarantined or only a fragment was caught up in the Sandbox which disappeared when I deleted the Sandbox. I also run in a non-admin windows profile most all the time but having Sandboxie run with reduced rights has always prevented malware from executing if it wasn't quarantined in time. Plus I am quite sure that much of the incoming malware are in the form of compressed HTTP packets all of which are destroyed before getting to my LAN. I'm conservative as I don't use Remote Access or Cloud services which why I don't yet need a VPN.

 

So, I will monitor my memory values for a few days to see if they get out of bounds. I also should mention that I am using the same version .6315 that you first noticed your issue. Also, I can be very critical when it comes to software but in general I am very pleased with EAM and my system is running very fast and stable as long as I don't download any bad Microsoft patches(I'm selective) and as long as  I continue to avoid the Win 10 nagware. I like the fact that EAM downloads the signatures to my OS and doesn't use the cloud(deal breaker) to process my files. I also read their recent article about memory usage and it made sense to me. I'm not sure what normal memory usage values are for a2service but my values do seem a bit high compared to the other processes. On the other hand, it makes sense that an EAM(anti-malware) type program would be using the most memory. Also, I have the option under EAM settings to "activate memory usage optimization" unchecked which will increase my physical memory usage and increase EAM processing speed, but 53% physical memory at 1st glance seems like a lot since I am not running any VM's and I have 16 GB of memory. I mention VMware because I believe you mentioned that you were running a VM.  I suspect that the huge amount of signatures that exist these days is the reason why most venders now use the cloud to process your files instead of loading the signatures on the users OS. It is likely that Emsisoft is doing a good job with memory usage considering how many signatures are processed, while other venders just gave up and went to the cloud. I am willing to deal with some increased memory usage in order to avoid cloud processing(I am old school and plan to die that way).

 

I have considered using VMware Workstation but I'm not sure that I would really have a need for it.(just something I wanted to get familiar with). I have successfully (many times) used Acronis for Restoration mostly for Bad MS Update Patches. I originally thought a VM setup would be advantageous for restoration but VM's do come with their own set extra complications and a good restoration program seems much simpler than VM, plus I do not perform a lot of testing.  I plan to lock down Win 7 in the year 2020 and run Linux Mint with Win 7 as a dual boot instead going to Win 10 or using VMware. I came to the conclusion because it just seemed much simpler. On the other hand, I'm already at 53% physical memory usage which to me is not that good if I were to install VMware Workstation with a guest OS. I do remember that my memory usage several months ago before I installed EAM was well below 53% physical usage. So, although I am concerned, I have not noticed any performance problems at the moment and I'm mostly bothered because I am already above the 50% physical memory usage point without even using a VM!!! 

 

Also, I thought I should point out(tangent) that in 2020 after Emsisoft stops supporting Windows 7, my having an External anti-virus(Kaspersky signatures) and IDP on the External Security Firewall/Gateway will allow me to continue protecting Win 7 and the Linux Mint OS's. At least that's how I hope everything goes. The only reason that I went off on so many tangents was to put a few extra ideas out there that I have found useful in order to help others find additional ways to protect their OS. However the External Firewall/Security Gateway will cost you some $, but is well worth it if set up to it's full potential.

 

What are your thoughts about the memory values that I posted?

Thanks

[GT500 645]

> If you make a backup of your settings, and then reset everything back to factory defaults, does this issue still happen?

I'm not going to try that yet.  I'd like to see if we can find out what's grabbing the storage.  As it is, the reboot I felt forced to do last night in the interest of system

(especially FS) integrity, might have lost us the chance to find out, but fortunately (or not, depending on your point of view) that looks not to be the case.

Also... I've not changed any settings in EIS in the last few days, apart from - now before two boots ago - having had debug logging on for a while.

You have a lot of non-standard settings, and the issue could be related to that. In that case, we might not be able to reproduce the issue without knowing what setting caused the issue (such as larger than standard log size for instance). If I can give our QA team something they can reproduce and test, fixing this will be faster than if I just send them logs.

[JeremyNicoll 58]

Yilee:

> I also could use Sysinternals process Explorer if you believe it would be much better for
> this situation.

Task Manager should be fine


> My physical memory usage is at 53 to 54%. A2servise uses the most memory out of the running
> processes at an average of 5,800,200 K for working set, peak working set and private working
> set with commit at around 6,375,656 K.   I have been booted for about 2 whole days.

Without knowing what's normal on your machine it's hard to be sure, but with no VMs and (as you
said lower down) 16 GB of RAM, on a system that's only been up two days, having commit charge
already at 6 GB seems wildly high to me.

Also are those working set figures for the whole OS?  I just rebooted (because I've shifted from
one house to another and chose to turn the laptop off completely while doing that) and although
- again - figures are climbing - working set for a2service here is currently only about 3 MB, not
5.8 GB.  I just happened to be watching as a set of updates arrived and working set went up to
about 180 MB while a2service dealt with that, but fell a few minutes later to about 3 MB again.


> On the other hand, it makes sense that an EAM (anti-malware) type program would be using the
> most memory.

It's sensible that it should use RAM if it needs it, but remember that most of Emsi's customers
won't have systems with 16 GB RAM in them, so it ought to be able to run in much less.   And in
that blog post of theirs, they said they can cram all those signatures into about 200 MB of RAM,
so that alone shouldn't be wasting GB of virtual memory.


> I mention VMware because I believe you mentioned that you were running a VM.

I'm not.  I used 'vm' and 'vs' in some posts, as abbreviations for 'virtual memory' and 'virtual
storage'.

[JeremyNicoll 58]

> You have a lot of non-standard settings

I'm not sure that I do, as not everything we have talked about has yet been translated into actual
settings changes.  I tend not to change stuff until/if I know I can devote some time in the
following days to watching the effects of those changes and/or finding ways to test their impact.

Since the problem seems to happen after reboots (from full 'cold' shutdowns), and as I haven't
touched any setting (apaert from turning off debug logging, and there's been several reboots

since then), I've copied all the .ini and ini.backup files out of my EIS install folder, used 7zip to

pack those up, and PMed that 7z file to you.

                                                                           

[Peter2150 45]

DId I get the impression you guys don't shutdown at night.   I have a win 7 x64 system, and I do run VMware VM's    and I never have a problem.  But I do shutdown at night.  Could this make a difference?

[stapp 131]

Peter on Win 8.1 and Win 10, shutdown is not the same as in Win 7.

 

It's sort of a suspend state (for want of a better explanation)

[Yilee 1]

 

Per Yilee

> My physical memory usage is at 53 to 54%. A2servise uses the most memory out of the running

> processes at an average of 5,800,200 K for working set, peak working set and private working

> set with commit at around 6,375,656 K.   I have been booted for about 2 whole days.

Without knowing what's normal on your machine it's hard to be sure, but with no VMs and (as you

said lower down) 16 GB of RAM, on a system that's only been up two days, having commit charge

already at 6 GB seems wildly high to me.

Also are those working set figures for the whole OS?  I just rebooted (because I've shifted from

one house to another and chose to turn the laptop off completely while doing that) and although

- again - figures are climbing - working set for a2service here is currently only about 3 MB, not

5.8 GB.  I just happened to be watching as a set of updates arrived and working set went up to

about 180 MB while a2service dealt with that, but fell a few minutes later to about 3 MB again.

 

 

This topic is getting interesting. Yes, the commit charge value and working set value as well as the other values except for the 53% physical memory usage were all specifically pertaining to a2service since that is the process that you are concerned with.  I was surprised also but wasn't sure because I never have investigated this type of problem.

 

This morning a2service values are in order are: 1.working set = 6,681,256 K  2. peak working = 6,681,276  K 3. private memory = 6,659,076 K and 4. Commit Size = 7,296,388 K and the total physical memory use is at 57% up from 53% yesterday evening. That's a big overnight jump !!! Now this phenomenon is leading me to these conclusions and I would like your opinion:

 

1. If someone like myself who much of the time for specific reasons prefers not to re-boot no more than once a month usually during MS Patch updates. It is very possible that my system could occasionally experience a crash once the physical memory usage became maxed out without my knowing always when the system was resuming from sleep. I do have this happen about every 2 weeks without any consistent reasons or crash codes and other diagnostic data.

               *this is despite the OS passing the following diagnostic test:  Memtest, Chkdsk, SFC, Windows System Readiness Tool, Driver Verification running for 4 days with around   30 Cyberfox(Firefox alternative) tabs opened, but 4 days may not be long enough to cause the memory crash during the Driver Verification Test.

 

2. It also occurred to me that EAM and EIS both are regularly subjected to update reboots and more lately program restarts which most likely both reset the memory values to their lowest starting values after which they start to grow again. I never have used a program that requires so many Build updates on such a regular basis! I have complained before about the inconvenience of having to reboot for users that need to keep their systems running for long periods of time. It would be one thing to have build updates say 6 times a year but every 2 weeks makes me suspicious that the constant build updates are a cover-up to have the memory values reset regularly before they crash too many users.  It would be OK with me if that was the case and if I knew that the program needed to be reset once a week with a Program Restart Only Not a RE-BOOT. Firefox/Cyberfox are programs that supposedly still need to have their RAM Cache cleared every so often due to memory usage issues and require program restarts. I just want to get everything out on the table so that I can  knowingly manage the situation appropriately.

 

3. Don't get me wrong, I still like the product because of the great customer service follow-up, the overall design, speed and for not processing our files in the cloud.

 

4. Concerning the EAM setting option "activate memory usage optimization" I suspect they use an algorithm based partly on how much physical memory is present on a machine. So if you have lots of unused physical memory and have the above option unchecked like I do, then maybe the program is going overboard and the code concerning the algorithm needs to be addressed. 

 

5. I suppose after I finish monitoring my a2 service memory values to see how high they will go, I may try running the EAM program with the option checked so that EAM will use the Paging File but I suspect that will lead to other issues when pressing the OS with many Cyberfox windows and tabs and/or navigating quickly back and forth with constant tab refreshing.

What do you thing about my new overnight values and my other speculations?

 

Update 042116 at 740pm Central us:

 

Its been just about 4 whole days since my last re-boot and since my last EAM update. Still running build 6315. My memory values continue to grow quite fast. Just finished some quite heavy video surfing followed by a full custom scan. My Task Manager Memory values are as follows:

 

*physical memory % = 75% but had peaked at 79% during the EAM scan.

* a2sevice working set memory = 9,585,900 K or 9.14 GB

* a2service commit size = 10,327,689 K or 9.84 GB

 

So it seems that the above values will continue to grow until my OS will BSOD unless I reboot or had a way to restart the EAM Program. My Laptop with similar setup right after reboot  had values that were normal like you had mentioned earlier about your machine after reboot, but it is also slowly showing the same memory value gains concerning a2service. I don't use it as much or as heavy.  I am assuming that if I rebooted this system the values would be normal at 1st also but I intend on running this test till the OS blue screens to make sure what the end result will be.

 

Suggestion: Seems like a Restart Program menu tab would be a simple way to deal with this problem while the memory usage gain problem is improved upon.

 

It seems this issue may turned out to be a serious problem for those of us who like to run without rebooting for several weeks at a time. I wonder how the Enterprise Users are dealing with this problem when using delayed updates for several weeks without rebooting.

 

What's your take on the whole matter and the suggestion? Firefox has a restart program menu item.

[Peter2150 45]

Peter on Win 8.1 and Win 10, shutdown is not the same as in Win 7.

 

It's sort of a suspend state (for want of a better explanation)

 

I know, but occasionally when the win 7x64 systems are squirrelly,  a reboot doesn't fix it, but a complete shutdown does, so something similiar is going on.

[stapp 131]

I know, but occasionally when the win 7x64 systems are squirrelly,  a reboot doesn't fix it, but a complete shutdown does, so something similiar is going on.

 

If you have debug logging turned on, you can see that a2service logs cannot be deleted like a2start and a2guard can be as it will say 'cannot delete as it is in use'  This is because a2service holds it state across the pc shutdown on Win 10/8. The a2service logs will contain all the logs since last system 'restart'  A2service logs prior to a 'restart' can be deleted.

 

How I wish Microsoft had not changed the terminology/technology for this as it opened a can of worms for software devs and users alike.

[GT500 645]

I'm not sure that I do, as not everything we have talked about has yet been translated into actual settings changes.  I tend not to change stuff until/if I know I can devote some time in the following days to watching the effects of those changes and/or finding ways to test their impact.

Perhaps, but something is different on your computer, as I'm not seeing this issue in my own testing.

[Yilee 1]

 

Update 042116 at 740pm Central us:

 

Its been just about 4 whole days since my last re-boot and since my last EAM update. Still running build 6315. My memory values continue to grow quite fast. Just finished some quite heavy video surfing followed by a full custom scan. My Task Manager Memory values are as follows:

 

*physical memory % = 75% but had peaked at 79% during the EAM scan.

* a2sevice working set memory = 9,585,900 K or 9.14 GB

* a2service commit size = 10,327,689 K or 9.84 GB

 

So it seems that the above values will continue to grow until my OS will BSOD unless I reboot or had a way to restart the EAM Program. My Laptop with similar setup right after reboot  had values that were normal like you had mentioned earlier about your machine after reboot, but it is also slowly showing the same memory value gains concerning a2service. I don't use it as much or as heavy.  I am assuming that if I rebooted this system the values would be normal at 1st also but I intend on running this test till the OS blue screens to make sure what the end result will be.

 

Suggestion: Seems like a Restart Program menu tab would be a simple way to deal with this problem while the memory usage gain problem is improved upon.

 

It seems this issue may turned out to be a serious problem for those of us who like to run without rebooting for several weeks at a time. I wonder how the Enterprise Users are dealing with this problem when using delayed updates for several weeks without rebooting.

 

What's your take on the whole matter and the suggestion? Firefox has a restart program menu item.

 

Update 4-22-16: starting 5th day since last reboot and last EAM update, still on Build 6315.

 

Woke up after no additional activity to see the following values in Task Manager. Also, I do not have the Debugger turned on and I do occasionally for good measure completely shutdown my machine and unplug it and hit the power button while it is unplugged, so the machine does occasionally get it's garbage wiped completely from RAM/ROM/Bios.

 

Task Manager Memory Values are as follows;

* Physical Memory = 80%

*a2service working set memory = 10,437,000 K

*a2service commit size = 11,215,428 K

I will let the readers convert to GB's. I'm sure that everyone who is following this article are already certain that my machine will blue screen in the next few days, probably during the weekend. What answers will be forthcoming.

 

I believe that JeremyNicoll stated that he had to perform a reboot due to moving his computer, so basically he will probably restart the test and will soon catch up to where I am at.

 

The only other useful information that I have noticed is that the buildup in the a2service memory values is directly tied to  how much activity you perform on the said computer as my laptop is gaining but at a lesser pace than this machine as I use it much more.

 

This issue it seems is here to stay until it is dissected and answers are forthcoming.

Thanks and I'm hoping to hear from JeremyNicoll when he can find time again. His insight is very helpful.

[JeremyNicoll 58]

Yilee, see contents of a message I just sent you.

[Yilee 1]

Yilee, see contents of a message I just sent you.

I understand. Please don't worry about this issue until you are caught up and ready. I'm in no hurry for now, at least until my computer crashes. I will continue to post updates and after it crashes I may have to open up my own thread. Thanks

[GT500 645]

I guess we'll just get debug logs then:

• Open Emsisoft Internet Security from the icon on your desktop.
• In the 4 little gray boxes at the bottom, move your mouse into the one that says Support, and click anywhere in that gray box.
• At the bottom, turn on the option that says Enable advanced debug logging.
• Either click on Overview in the menu at the top, or close the Emsisoft Internet Security window.
• Reproduce the issue you are having.
• Once you have reproduced the issue, open Emsisoft Internet Security again, and click on the gray box for Support again.
• Click on the button that says Send an email.
• Select the logs in the left that show today's dates.
• Fill in the e-mail contact form with your name, your e-mail address, and a description of what the logs are for (if possible please leave a link to the topic on the forums that the logs are related to in your message).
• If you have any screenshots or another file that you need to send with the logs, then you can click the Attach file button at the bottom (only one file can be attached at a time).
• Click on Send now at the bottom once you are ready to send the logs.

Important: Please be sure to turn debug logging back off after sending us the logs. There are some negative effects to having debug logging turned on, such as reduced performance and wasting hard drive space, and it is not recommended to leave debug logging turned on for a long period of time unless it is necessary to collect debug logs.

Please note that if you have a lot of debugs logs, then you should not send all of them. There is a size limit, and currently there is no error if the message is rejected due to the size being too large. Normally we only need one copy of the 4 or 5 different logs that have been saved after the time you reproduced the issue (the list shows what time each log was saved). Those logs have the following names:

• Security Center
• Protection Service
• Real-Time Protection
• Firewall
• Logs database (contains the logs you can view in Emsisoft Internet Security by clicking on Logs at the top of the window).

[Yilee 1]

I guess we'll just get debug logs then:

• Open Emsisoft Internet Security from the icon on your desktop.
• In the 4 little gray boxes at the bottom, move your mouse into the one that says Support, and click anywhere in that gray box.
• At the bottom, turn on the option that says Enable advanced debug logging.
• Either click on Overview in the menu at the top, or close the Emsisoft Internet Security window.
• Reproduce the issue you are having.
• Once you have reproduced the issue, open Emsisoft Internet Security again, and click on the gray box for Support again.
• Click on the button that says Send an email.
• Select the logs in the left that show today's dates.
• Fill in the e-mail contact form with your name, your e-mail address, and a description of what the logs are for (if possible please leave a link to the topic on the forums that the logs are related to in your message).
• If you have any screenshots or another file that you need to send with the logs, then you can click the Attach file button at the bottom (only one file can be attached at a time).
• Click on Send now at the bottom once you are ready to send the logs.

Important: Please be sure to turn debug logging back off after sending us the logs. There are some negative effects to having debug logging turned on, such as reduced performance and wasting hard drive space, and it is not recommended to leave debug logging turned on for a long period of time unless it is necessary to collect debug logs.

Please note that if you have a lot of debugs logs, then you should not send all of them. There is a size limit, and currently there is no error if the message is rejected due to the size being too large. Normally we only need one copy of the 4 or 5 different logs that have been saved after the time you reproduced the issue (the list shows what time each log was saved). Those logs have the following names:

• Security Center
• Protection Service
• Real-Time Protection
• Firewall
• Logs database (contains the logs you can view in Emsisoft Internet Security by clicking on Logs at the top of the window).

 

Thank You for your support offer. Over the weekend the following Happened:

1.With my Physical memory at around 85% and with a2service with even higher memory values than what I previously posted the system finally did crash but not with a BSOD. I woke up to find the computer unresponsive to the keyboard and mouse and a black monitor. The system fans were still running which indicates that the OS did not enter it's usual sleep mode.

 

2. After hitting the reset button I  performed a chkdsk in safe mode and then a SFC /scannow, both were ok. Since the OS did not formally crash I don't have a full memory crash dump to analyze, which I would have if it had BSOD'ed.

 

3. The only significant info that made any sense to me was the following:

   a. The improper shutdown  happened within a minute of my OS completely finishing an Acronis Full Backup instead of a Differential which takes up much more memory resources than does Differential and only happens around every 12 days per schedule.

   b. Just based on my experience and my speculation the way in which the system was unable to immediately go back to sleep after the Acronis backup as per schedule indicates that it most likely ran out of memory resources while initiating the idle/sleep routine. If it just ran out of system memory in the middle of this routine, the way that I found it make sense that it was unresponsive with no BSOD. I have my OS setup to stay on any BSOD and not to auto reboot.

 

4. So, the system was manully rebooted and now all of the memory values are back to normal and have only slighty grown but I have not been using the computer much.  My other computer(laptop) is now at 40 % physical memory use and the only Process that continues to grow larger and larger is a2service. After reboot this computer started out at 13% physical memory but in 2 days is up to 18% due to a2service memory values continuing to grow.

 

So, what would you have me do for now?

 

I'm still running build .6315 which is OK with me as I employ other external anti-malware at the gateway. Would you prefer that I update 1st and get back to you if a2service's memory values get up to 4GB's usage range and then turn on the Debugger or just stay with  build .6315 and wait the same way. I prefer to debug it once its' close crashing again or maybe the problem might disappear.

 

What approach would you like to take?

[GT500 645]

Have you checked your Event Logs to see if there were any errors logged when the issue happened? You can open the Event Viewer to see the logs by right-clicking on My Computer, selecting Manage, double-clicking on Event Viewer in the list on the left, double-clicking on Windows Logs, and selecting Application or System to see the relevant logs (the other categories probably won't have any information about the issue at hand).

Which memory values are growing? The Private Bytes or the Working Set (assuming you're using Process Explorer to view this), or perhaps something else?

Does turning the option that says "Activate memory usage optimization" on or off in the settings have any effect on the issue?

Does resetting the settings to factory defaults have any effect on the issue?

[Peter2150 45]

Just so I can test, is this problem due to leaving the machine on 24/7  If so I can test.

[GT500 645]

This is essentially what I see of the memory usage for a2service.exe every time I check (I have the memory usage optimization turned off):

Download Image

I do shut my computer off every day, so if there is a memory related issue that develops slowly over time I wouldn't notice. That being said, there are currently no other similar reports of issues, so I suspect it may have something to do with configuration.

[Yilee 1]

Just so I can test, is this problem due to leaving the machine on 24/7  If so I can test.

Yes. this issue would only apply to those users who for whatever reason prefer to leave their machine running for up to 2 to 4 weeks at a time and avoid reboots except for when monthly MS update patches are installed. Reboots will reset a2service memory values back to normal. I am not sure about EAM or EIS program restarts at this time. I plan to test that later when I update to the next build. I'm currently giving my machine a few days to elevate the a2service memory values before I update to the newest stable build so that I can see if a2service memory values return to normal ranges.  The issue is also affected by how much the machine is used and the type of tasks performed. Task such as streaming video, acronis backups and heavy surfing real-time charting with many windows and tabs open will cause all of the a2service memory values to rise quickly. In my case it takes about 6 to 14 days to crash the machine without causing a BSOD and no obvious driver errors depending on how much heavy use I put the machine through. See my previous post above for details.

[Yilee 1]

Have you checked your Event Logs to see if there were any errors logged when the issue happened? You can open the Event Viewer to see the logs by right-clicking on My Computer, selecting Manage, double-clicking on Event Viewer in the list on the left, double-clicking on Windows Logs, and selecting Application or System to see the relevant logs (the other categories probably won't have any information about the issue at hand).

Which memory values are growing? The Private Bytes or the Working Set (assuming you're using Process Explorer to view this), or perhaps something else?

Does turning the option that says "Activate memory usage optimization" on or off in the settings have any effect on the issue?

Does resetting the settings to factory defaults have any effect on the issue?

In my case it goes without saying that I always check all logs available with or without having a complete memory crash dump report (i use WhoCrashed to analyze and keep running records). When I say all logs I am referring to all Windows event logs and all Application and Sevices Logs/Microsoft/Windows logs looking for clues that I can put together around the time of the crash event. I even Review the Reliability Report inside the Action Center GUI.

Everything points to the fact that after the resource intensive Full Acronis Backup Successfully completed(validated) the system entered the idle state(per event log) and began the partial stopping of services(didn't finish) in order to enter the sleep phase. However it must have ran out of memory resources and became paralyzed because its memory resources became depleted during midstream. My guess is that if I had been there to observe, it is likely that i would have seen some memory depletion warnings during the acronis backup. But the backups are performed while I am sleeping.

 

As far as a2service's memory values are concerned, in my previous post I stated that I enabled all of the memory value column options in Task Manager(working set,peak working,private working,commit size, paged pool, NP Pool) and also kept tabs on my physical memory % usage. Just before the improper shutdown/sleep/resume lockup/crash occured all of the a2service memory values were each elevated in the 6 to 7 GB range and my physical memory usage the night before the system freeze/crash was at 81 %.

 

So, you can always assume that I have always looked at all of my logs.

 

So the question remains, do you want me to perform the debugging process on build 6315 or would you prefer I use the next stable build? 

[GT500 645]

As far as a2service's memory values are concerned, in my previous post I stated that I enabled all of the memory value column options in Task Manager(working set,peak working,private working,commit size, paged pool, NP Pool) ...

I guess I should ask, is the Task Manager showing the same values as Process Explorer? It's generally recommended not to use the Task Manager to monitor memory usage, as utilities such as Process Explorer and Process Hacker are widely considered to be more accurate. If the Task Manager is showing the same values, then it shouldn't be a big deal.

So the question remains, do you want me to perform the debugging process on build 6315 or would you prefer I use the next stable build?

You can use the latest stable build. It actually doesn't matter, if it's happening with the beta and the stable then you can get debug logs from either.

[JeremyNicoll 58]

Periodically in the last couple of weeks, I've cross-checked Process Hacker's figures with that of Task Manager (in W8.1), and here, they've always agreed with each other.

 

It's interesting that Yilee sees the problem on W7 Ultimate (on a desktop?) and on a laptop (precise OS unstated above, maybe W7?), and I on W8.1, both of us with 64 bit

systems. 

 

Also, my machine has "activate memory usage optimization" turned on, and has done ever since I installed EIS on 28th Feb 2016 (I looked back to my install notes where

I'd noted the settings I chose to start with).  So if that setting is relevant, it was harmless until the reported start of this problem. 

 

I did a full shutdown & reboot again over the weekend, but this time turned debug logging on just before I shut the system.  Possibly later today, depending on how high the

memory use has climbed to (it's doing it steadily but I've not been awake so much in the last few days), I'll shutdown & reboot and then make the debug logs for the whole

time: boot through to shutdown  available to you.

[JeremyNicoll 58]

If there's a way to, and if it would help your developers, I'd be willing to install an older build to see if the problem goes away.

[Yilee 1]

I guess I should ask, is the Task Manager showing the same values as Process Explorer? It's generally recommended not to use the Task Manager to monitor memory usage, as utilities such as Process Explorer and Process Hacker are widely considered to be more accurate. If the Task Manager is showing the same values, then it shouldn't be a big deal.

You can use the latest stable build. It actually doesn't matter, if it's happening with the beta and the stable then you can get debug logs from eitherSin

Since my system crashed I have not seen the same previous aggressive move upward in a2service memory values so there could be an unknown mitigating factor that is no longer present. Therefore I am doing the following:

*will update to latest stable build

*getting rid of AMD's catalyst and switching to their new Radeon Crimson Control(does not use ccc.exe or mom.exe,never haved liked those processes). The switch went over without any snags. I used AMD's clean uninstall utility after regular uninstall.

*If the problem with a2service's elevated memory values returns I will open a new thread or join back on this one if it is still open and applicable. For now on I will just actively monitor the values and hope that the mitigating factor is removed either by AMD driver updates or MS's monthly Patches. Going forward, many of us who want to remain on Windows 7 will be facing a struggle to combat many of the important and recommended patches that will in the end make Win 7 have more compatibility problems. Best to choose your patches wisely and read the fine print and take notes. I just saying that most unknown mitigating factors will mostly be a result of Monthly MS patches and in many cases Graphics Drivers  with full software packages.

*To Jeremy: As you may recall, I did not have that "memory optimization" enabled in EAM settings. Also, if it happens again I will use Process Explorer and Task Manager just to be sure.

*I wish everyone luck and will be following this thread.

*Also my laptop with 8GB of physical memory has stabilized at 34% physical memory usage and has not gained any further. Win7 Pro with same EAM settings. It never did reach catastrophic values.

 

Thanks

[GT500 645]

If there's a way to, and if it would help your developers, I'd be willing to install an older build to see if the problem goes away.

Getting a hold of an old build isn't difficult, however keeping it is. The updater will automatically download and install the latest version unless you disable updates.

[JeremyNicoll 58]

> Getting a hold of an old build isn't difficult, however keeping it is. The updater will automatically
> download and install the latest version unless you disable updates.

And that would also prevent signature updates?   I wouldn't want to do that for very long.
                                                                                         

[JeremyNicoll 58]

Arthur, see PM for a full set of debug logs for 24th-29th, and a summary of climbing memory use over that period.

[GT500 645]

And that would also prevent signature updates?   I wouldn't want to do that for very long.

Correct.

Arthur, see PM for a full set of debug logs for 24th-29th, and a summary of climbing memory use over that period.

Thank you. I have forwarded them to our QA Manager, and he will look into this.

[JeremyNicoll 58]

I just noticed something else, which might be a clue - I've no idea.  Sysinternals' pslist shows nearly 160 threads in a2service

(I can see that might be normal if many of them represent hooks in other processes, but if not, it seems an awful lot), and far

far fewer in a2guard and a2start.

 

C:\>pslist -x a2

pslist v1.3 - Sysinternals PsList
Copyright © 2000-2012 Mark Russinovich
Sysinternals - www.sysinternals.com

Process and thread information for SAMSUNG-NP350:

Name                Pid      VM      WS    Priv Priv Pk   Faults   NonP Page
a2service          1176 1793944   10348  971332 1016940  7380062     86  286
 Tid Pri    Cswtch            State     User Time   Kernel Time   Elapsed Time
1180  10      1183     Wait:UserReq  0:00:02.687   0:00:00.546    6:59:14.175
1296  14       650     Wait:UserReq  0:00:00.031   0:00:00.015    6:59:13.550
1300   9      2627     Wait:UserReq  0:00:00.046   0:00:00.031    6:59:13.550
1304   8     25523   Wait:DelayExec  0:00:00.000   0:00:00.000    6:59:13.550
1308   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.535
1312  10   2240052     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.504
1316   8   2239472     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.504
1320   9     34427     Wait:UserReq  0:00:00.031   0:00:00.031    6:59:13.363
1324   9   2070425   Wait:DelayExec  0:00:00.031   0:00:00.015    6:59:13.363
1328   9         8     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.363
1336   8         3     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.363
1340  10      1244     Wait:UserReq  0:00:00.546   0:00:00.453    6:59:13.363
1344   8    244279   Wait:DelayExec  0:00:00.000   0:00:00.000    6:59:13.363
1356   8       782     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:13.003
1408   9         3     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:12.957
1412   1       461     Wait:UserReq  0:00:00.000   0:00:00.312    6:59:12.957
1436   9    131616     Wait:UserReq  0:00:00.625   0:00:00.109    6:59:12.332
1440   9    185999     Wait:UserReq  0:00:00.859   0:00:00.156    6:59:12.332
1444   9     10151     Wait:UserReq  0:00:00.046   0:00:00.015    6:59:12.332
1448   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:12.300
1452   8     13834     Wait:UserReq  0:00:00.031   0:00:00.140    6:59:11.877
1460   9      2724     Wait:UserReq  0:00:04.796   0:00:00.187    6:59:11.877
1468   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:11.877
1472  10        25     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:11.877
1476  10     75543     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:11.877
1504   9      2936     Wait:UserReq  0:00:04.609   0:00:00.140    6:59:11.862
1824   8     50557     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:09.625
2352   9      2947     Wait:UserReq  0:00:04.468   0:00:00.093    6:59:07.909
3696   8        12     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:00.167
3700   8         3       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3704   8      1550       Wait:Queue  0:00:01.296   0:00:00.125    6:59:00.167
3708   8      1157       Wait:Queue  0:00:00.812   0:00:00.015    6:59:00.167
3712   9        23       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3716   9        31       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3720   9        49       Wait:Queue  0:00:00.015   0:00:00.000    6:59:00.167
3724   8      2548       Wait:Queue  0:00:01.718   0:00:00.203    6:59:00.167
3728   8       520       Wait:Queue  0:00:00.375   0:00:00.015    6:59:00.167
3732   9     14121       Wait:Queue  0:00:04.546   0:00:01.234    6:59:00.167
3736   9        18       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3744   9         9     Wait:UserReq  0:00:00.000   0:00:00.000    6:59:00.167
3748   8         3       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3752   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3756   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3764   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3768   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3772   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3776   8         1       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3780   8         2       Wait:Queue  0:00:00.000   0:00:00.000    6:59:00.167
3784   8       342       Wait:Queue  0:00:00.000   0:00:00.015    6:59:00.167
3788   9      1337       Wait:Queue  0:00:00.015   0:00:00.078    6:59:00.167
3792  10     15926     Wait:UserReq  0:00:00.312   0:00:00.781    6:59:00.167
4164   8         4     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:54.610
4172  10      6912     Wait:UserReq  0:00:00.031   0:00:00.015    6:58:54.579
4192   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:54.286
4212   9       912     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:53.770
4216   8       863     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:53.770
5468  10     20074     Wait:UserReq  0:00:01.984   0:00:00.703    6:58:33.048
 244   9      3178     Wait:UserReq  0:00:04.203   0:00:00.125    6:57:09.445
2976   9      2814     Wait:UserReq  0:00:04.078   0:00:00.093    6:57:09.445
2852   9      2470     Wait:UserReq  0:00:03.906   0:00:00.109    6:57:09.445
4060   9      2260     Wait:UserReq  0:00:03.718   0:00:00.093    6:56:58.667
2760   9      2413     Wait:UserReq  0:00:03.562   0:00:00.125    6:56:58.664
3028   8         3     Wait:UserReq  0:00:00.000   0:00:00.000    6:52:44.120
3524   8         2     Wait:UserReq  0:00:00.000   0:00:00.000    6:52:41.500
4568   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:52:41.473
4888   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:52:41.447
1880   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    6:52:41.421
 776   9      2060     Wait:UserReq  0:00:02.281   0:00:00.125    6:23:21.828
3976   9      1887     Wait:UserReq  0:00:02.312   0:00:00.046    6:23:21.826
5792   9      2035     Wait:UserReq  0:00:02.125   0:00:00.015    6:17:09.031
1704   9      1625     Wait:UserReq  0:00:01.906   0:00:00.031    6:00:35.519
5612   9      1781     Wait:UserReq  0:00:01.937   0:00:00.046    6:00:35.271
1772   9      1900     Wait:UserReq  0:00:01.890   0:00:00.046    6:00:35.266
5532   9      1587     Wait:UserReq  0:00:02.015   0:00:00.031    6:00:35.264
2676   9      2111     Wait:UserReq  0:00:02.140   0:00:00.031    6:00:35.263
3216   9      1633     Wait:UserReq  0:00:01.812   0:00:00.015    6:00:35.240
4268   9      1658     Wait:UserReq  0:00:01.734   0:00:00.015    6:00:33.103
 124  10      1953     Wait:UserReq  0:00:01.890   0:00:00.062    6:00:33.101
1924   9      1618     Wait:UserReq  0:00:01.453   0:00:00.046    6:00:33.077
5712   9      1728     Wait:UserReq  0:00:01.796   0:00:00.062    6:00:32.996
5496   9      1611     Wait:UserReq  0:00:01.531   0:00:00.046    6:00:32.981
3540  10      1847     Wait:UserReq  0:00:01.828   0:00:00.093    6:00:32.746
4644   9      1823     Wait:UserReq  0:00:01.843   0:00:00.015    6:00:32.738
6100   9      1643     Wait:UserReq  0:00:01.703   0:00:00.031    6:00:32.738
4564   9      1698     Wait:UserReq  0:00:02.187   0:00:00.046    6:00:32.735
3156   9      1727     Wait:UserReq  0:00:01.734   0:00:00.015    6:00:32.734
4464   9      1664     Wait:UserReq  0:00:01.703   0:00:00.046    6:00:32.732
3600   9      1730     Wait:UserReq  0:00:01.718   0:00:00.031    6:00:32.731
2000   9      1680     Wait:UserReq  0:00:01.468   0:00:00.015    6:00:32.723
5796  10      1436     Wait:UserReq  0:00:01.687   0:00:00.046    6:00:32.720
4932   9      1612     Wait:UserReq  0:00:01.718   0:00:00.046    6:00:32.720
1248   9      1886     Wait:UserReq  0:00:01.843   0:00:00.000    6:00:32.680
1076   9      1579     Wait:UserReq  0:00:01.656   0:00:00.062    6:00:32.674
4532   9      1594     Wait:UserReq  0:00:01.750   0:00:00.031    6:00:32.648
5084   9      1852     Wait:UserReq  0:00:02.156   0:00:00.015    6:00:32.644
2876   9      1690     Wait:UserReq  0:00:01.578   0:00:00.031    6:00:32.615
2188   9      1412     Wait:UserReq  0:00:01.890   0:00:00.031    6:00:32.612
5852   9      1578     Wait:UserReq  0:00:02.078   0:00:00.015    6:00:32.582
4944   9      1381     Wait:UserReq  0:00:01.750   0:00:00.031    6:00:32.488
2668   9      1508     Wait:UserReq  0:00:02.000   0:00:00.015    6:00:32.486
3860   9      1524     Wait:UserReq  0:00:01.546   0:00:00.046    6:00:32.484
3820   9      1829     Wait:UserReq  0:00:01.921   0:00:00.015    6:00:32.483
5668   9      1773     Wait:UserReq  0:00:01.890   0:00:00.031    6:00:32.481
1352   9      1950     Wait:UserReq  0:00:01.906   0:00:00.000    6:00:32.480
4112   9      1289     Wait:UserReq  0:00:01.843   0:00:00.046    6:00:32.397
4412   9      1407     Wait:UserReq  0:00:01.906   0:00:00.015    6:00:32.317
5696   9      1282     Wait:UserReq  0:00:01.828   0:00:00.078    6:00:31.234
5568   9      1564     Wait:UserReq  0:00:01.750   0:00:00.031    6:00:31.101
5596   9      1300     Wait:UserReq  0:00:01.312   0:00:00.031    5:57:28.019
 348   9       804     Wait:UserReq  0:00:01.125   0:00:00.031    5:57:28.019
2252   9      1030     Wait:UserReq  0:00:01.062   0:00:00.031    5:57:27.998
1628   9       792     Wait:UserReq  0:00:01.421   0:00:00.000    5:57:27.997
2872   9      1148     Wait:UserReq  0:00:01.375   0:00:00.000    5:57:27.963
1040   9      1191     Wait:UserReq  0:00:01.453   0:00:00.046    5:57:27.962
5336   9       803     Wait:UserReq  0:00:01.265   0:00:00.015    5:57:27.952
1208   9      1087     Wait:UserReq  0:00:01.203   0:00:00.031    5:57:27.952
2204   9      1101     Wait:UserReq  0:00:01.109   0:00:00.031    5:57:27.850
6132   9      1043     Wait:UserReq  0:00:01.203   0:00:00.046    5:57:27.740
4436   9       892     Wait:UserReq  0:00:01.234   0:00:00.046    5:57:27.717
5536   9       833     Wait:UserReq  0:00:01.265   0:00:00.015    5:57:27.716
2232   9       958     Wait:UserReq  0:00:01.109   0:00:00.062    5:57:27.715
  92   9      1028     Wait:UserReq  0:00:01.234   0:00:00.015    5:57:27.714
1292   9      1240     Wait:UserReq  0:00:01.171   0:00:00.015    5:57:27.691
5972   9      1402     Wait:UserReq  0:00:01.140   0:00:00.031    5:57:27.690
3844   9      1352     Wait:UserReq  0:00:01.234   0:00:00.078    5:57:27.682
4256   9      1018     Wait:UserReq  0:00:01.265   0:00:00.031    5:57:27.582
1012   9       986     Wait:UserReq  0:00:01.203   0:00:00.015    5:57:27.484
2540   9      1688     Wait:UserReq  0:00:01.375   0:00:00.046    5:57:27.483
2644   9       886     Wait:UserReq  0:00:00.875   0:00:00.062    5:57:27.475
 920   9       963     Wait:UserReq  0:00:01.046   0:00:00.000    5:57:27.434
5564   9      1019     Wait:UserReq  0:00:01.203   0:00:00.031    5:57:27.379
5328   9       975     Wait:UserReq  0:00:01.296   0:00:00.031    5:57:27.372
5024   9      1054     Wait:UserReq  0:00:01.296   0:00:00.000    5:57:27.371
3288   9      1049     Wait:UserReq  0:00:01.203   0:00:00.062    5:57:27.350
5776   9       925     Wait:UserReq  0:00:01.281   0:00:00.031    5:57:27.346
 788   9       658     Wait:UserReq  0:00:01.140   0:00:00.000    5:57:27.345
2600   9      1196     Wait:UserReq  0:00:00.921   0:00:00.031    5:57:27.299
3244   9      1512     Wait:UserReq  0:00:01.375   0:00:00.031    5:57:27.232
3160   9       714     Wait:UserReq  0:00:01.171   0:00:00.000    5:57:27.114
5764  10      1045     Wait:UserReq  0:00:01.140   0:00:00.031    5:57:27.107
5916   9       887     Wait:UserReq  0:00:01.062   0:00:00.031    5:57:27.099
4908   9       848     Wait:UserReq  0:00:00.937   0:00:00.031    5:57:27.098
2424   9      1589     Wait:UserReq  0:00:01.296   0:00:00.093    5:57:27.096
5744   9      1326     Wait:UserReq  0:00:01.343   0:00:00.046    5:57:27.073
5484   9      1011     Wait:UserReq  0:00:01.171   0:00:00.015    5:57:27.051
2508   9       862     Wait:UserReq  0:00:01.156   0:00:00.062    5:57:27.048
3372   9       889     Wait:UserReq  0:00:01.093   0:00:00.015    5:57:27.035
4608   9       662     Wait:UserReq  0:00:01.109   0:00:00.015    5:57:27.035
2004   9       432     Wait:UserReq  0:00:00.281   0:00:00.015    4:25:35.622
5816   9       306     Wait:UserReq  0:00:00.406   0:00:00.000    4:25:35.548
 620   9       362     Wait:UserReq  0:00:00.359   0:00:00.000    4:25:35.501
6028   9       236     Wait:UserReq  0:00:00.468   0:00:00.000    4:25:35.415
3000   8         1     Wait:UserReq  0:00:00.000   0:00:00.000    1:00:43.815
3544   8         3     Wait:UserReq  0:00:00.000   0:00:00.000    1:00:43.815
3200   9       126       Wait:Queue  0:00:00.000   0:00:00.000    0:01:39.692
4632   8         4       Wait:Queue  0:00:00.000   0:00:00.000    0:01:39.692
 304   8         6     Wait:UserReq  0:00:00.000   0:00:00.000    0:00:30.892

Name                Pid      VM      WS    Priv Priv Pk   Faults   NonP Page
a2guard            4128  188088    7340   41912   51608   135607     46  304
 Tid Pri    Cswtch            State     User Time   Kernel Time   Elapsed Time
4132  12    362077     Wait:UserReq  0:00:00.718   0:00:00.906    6:58:55.557
4176  12     34603     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:54.579
4180   2       243     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:54.380
4196  10      5728     Wait:UserReq  0:00:00.140   0:00:00.328    6:58:54.286
  32   8         2       Wait:Queue  0:00:00.000   0:00:00.000    0:00:44.533
3360   8         1       Wait:Queue  0:00:00.000   0:00:00.000    0:00:14.438

Name                Pid      VM      WS    Priv Priv Pk   Faults   NonP Page
a2start            6072  213080    4564   58600   61880   456012     64  314
 Tid Pri    Cswtch            State     User Time   Kernel Time   Elapsed Time
6076  10    402826     Wait:UserReq  0:00:03.890   0:00:02.875    6:58:34.282
4264  10     26481     Wait:UserReq  0:00:00.015   0:00:00.015    6:58:33.048
4228   8         2     Wait:UserReq  0:00:00.000   0:00:00.000    6:58:33.032
4396   1       228     Wait:UserReq  0:00:00.000   0:00:00.015    6:58:33.032
5464   8   2064600   Wait:DelayExec  0:00:00.000   0:00:00.000    6:58:32.657
2284  10    241126   Wait:DelayExec  0:00:00.000   0:00:00.015    6:52:44.222
4940   8         1       Wait:Queue  0:00:00.000   0:00:00.000    0:00:14.500
 212   8         1       Wait:Queue  0:00:00.000   0:00:00.000    0:00:14.500


C:\>

 

The machine was last booted around 7 hours ago, which would (I suppose) correlate with the

near-7 hour elpased time value on some detail lines.

[GT500 645]

I just noticed something else, which might be a clue - I've no idea.  Sysinternals' pslist shows nearly 160 threads in a2service (I can see that might be normal if many of them represent hooks in other processes, but if not, it seems an awful lot), and far far fewer in a2guard and a2start.

That's because a2service.exe handles most of the functions of our software. Proper software design is to isolate many functions in their own threads so that if there's something being processed that takes a lot of CPU time, the rest of the application doesn't freeze while waiting on it.

[JeremyNicoll 58]

I understand the purpose of threading (and pre-emptive scheduling) perfectly thanks.  I was
just asking if you'd expect to see quite so many threads.  For all I know, there might be a
problem with threads that should have ended, and released their acquired memory, not doing
so.

Is there any progress from whoever's been looking at the debug logs I sent?

[GT500 645]

Is there any progress from whoever's been looking at the debug logs I sent?

I've talked to our QA Manager about the issue, however I don't think a developer has had a chance to look over the logs yet.

[JeremyNicoll 58]

OK.  In the last few days I've been using the machine less heavily than last week, however
one thing I have done is install all of last month's Windows Updates, which I'd put off
doing knowing that I'd very likely have to reboot while doing that.   Memory use is still
climbing though.

[Peter2150 45]

What is wrong with rebooting, and even shutting down on occasion.   Sorry, but to me this is beginning to sound like a self inflicted issue

[JeremyNicoll 58]

Peter: I'm lucky enough to have 8 GB of RAM in this machine, and therefore the machine can run for a few days without that.  If I had only, say 2 GB, I

would struggle to get one day's use.  Surely that's not reasonable these days.  Moreover, this memory problem started abruptly when EIS got updated

a few weeks ago.  Before that, no problem.  It's not /self-/inflicted.

 

If I were running any sort of server I would be appalled at the idea that a machine might need rebooted as frequently as you suggest.

[GT500 645]

This must be an isolated issue, because if this was happening on every computer most people would be reporting major issues. Whenever I check, the Private Bytes for a2service.exe is usually around 550MB, and doesn't seem to really deviate from that by very much.

Have we tried a fresh install to see if that has any effect on the issue?

[JeremyNicoll 58]

> This must be an isolated issue.... most people reporting major issues...

 

Apparently.   Unless vast numbers of your users reboot frequently, or have had the problem but don't realise it.

 

> Have we tried a fresh install to see if that has any effect on the issue?

 

Why should it?  What's different about a fresh install compared with one that's been updated by your released updates?  

[Peter2150 45]

 

Why should it?  What's different about a fresh install compared with one that's been updated by your released updates?  

 

Jeremy

 

I've been working with computers since before IBM introduced the PC, and working with PC's ever since.  One thing I've learned is half the time problems are resolved by throwing out the logical and going with the illogical.  So to me the answer to your question is simply because.

 

Here's what I would do.

 

1.  Image your system with a good imaging program, so you can get back to where you are if you need to.

2.  Install a trial or Revo uninstaller and completely uninstall EIS

3.  Completely shut down your system and power it off.  Essentially unplug it.

4.  Power up and Install the latest stable EIS release, and then let it update.

5.  Now test and see if the problem persists.   If it does now you can be sure it's EIS

 

Pete