onbox

AV-Comparatives Real-World Protection Test September 2018

Recommended Posts

23 hours ago, onbox said:

For us the best configuration of Emsisoft File Guard is THOROUGH

Why would that matter with the Behavior Blocker? It should have stopped anything the File Guard didn't. Does anyone know what version of EAM they tested?

Share this post


Link to post
Share on other sites
58 minutes ago, GT500 said:

Why would that matter with the Behavior Blocker? It should have stopped anything the File Guard didn't. Does anyone know what version of EAM they tested?

version 2018.7

Share this post


Link to post
Share on other sites

It looks like it was due to a Behavior Blocker issue. Our malware analysts and developers are aware of what went wrong.

Share this post


Link to post
Share on other sites
2 hours ago, Castor said:

Well, to see a problem is the first step. But more important: Is that issue fixed yet?

 

As far as I know it hasn't been, however I haven't been given any further information. I do know that AV-Comparatives shares undetected samples, so those are added for detection by the File Guard as soon as they are verified as legitimate misses. I also know that the issue isn't as common in the real world as it is in testing, meaning the odds of real users running into this issue is extremely small.

Share this post


Link to post
Share on other sites
On 10/17/2018 at 4:40 PM, GT500 said:

I do know that AV-Comparatives shares undetected samples, so those are added for detection by the File Guard as soon as they are verified as legitimate misses

That is not the point; this was a test and adding the detection after the test is pointless

 

On 10/17/2018 at 4:40 PM, GT500 said:

I also know that the issue isn't as common in the real world as it is in testing, meaning the odds of real users running into this issue is extremely small.

Again this was a test; how many times ,in real life you will have to solve a triple integral??? Yet, this could be a test for university admission.

 

On 10/17/2018 at 1:53 PM, GT500 said:

It looks like it was due to a Behavior Blocker issue

Behavior blocker is a mature product (since Mamutu) , which in theory doesn't require permanent maintenance; so, what issue????

Share this post


Link to post
Share on other sites
18 hours ago, andone said:

That is not the point; this was a test and adding the detection after the test is pointless

Adding detection after the test ensures that the threats are detected. This means that while EAM didn't catch the threats in the test, it won't miss them if customers run into them. This is one of the reasons why the better testing organizations share the samples with the anti-virus software vendors. And of course, to make sure we can fix any issues that led to detection failures.

 

18 hours ago, andone said:

Yet, this could be a test for university admission.

Is a 97% considered a fail for university admission?

 

18 hours ago, andone said:

Behavior blocker is a mature product (since Mamutu) , which in theory doesn't require permanent maintenance; so, what issue????

The Behavior Blocker does require periodic maintenance and updates. Threats do change over time, and we need to be able to make sure that underlying protection technology can be adapted to deal with them. If it were possible to compare Mamutu to the Behavior Blocker in Emsisoft Anti-Malware today, I think you'd find Mamutu far less capable.

Share this post


Link to post
Share on other sites
4 hours ago, GT500 said:

the better testing organizations share the samples with the anti-virus software vendors

The testing organizations share the samples with the "not so good antivirus software" because otherwise the vendor would complain about detection rate ; when a teacher discuses with a student the mark and shares the failed subjects , is not for the student to know in the future but rather to justify the mark.

 

4 hours ago, GT500 said:

Is a 97% considered a fail for university admission?

When the passing mark is 99%, yes , 97% seems low.

 

4 hours ago, GT500 said:

The Behavior Blocker does require periodic maintenance and updates

In over 2000 years we still have the same 10 commandments as  "behavior standard" , unmodified, and the expectation is for us to behave accordingly .

At least that was my understanding about a behavior blocker; once you have a set of rules, should be good for life....

Share this post


Link to post
Share on other sites
18 hours ago, andone said:

The testing organizations share the samples with the "not so good antivirus software" because otherwise the vendor would complain about detection rate

The better testing organizations do give vendors a chance to review samples to ensure that everything that was "missed" was a legitimate miss, however the testing organizations generally have finally authority over what is considered a "miss" and what isn't.

That being said, they do share samples after the tests to ensure detection as well.

 

18 hours ago, andone said:

when a teacher discuses with a student the mark and shares the failed subjects , is not for the student to know in the future but rather to justify the mark.

If a "teacher" does not bother to help a student understand something they do no, then have they not failed in their duty as a teacher?

Not that this comparison is relevant to the discussion. Testing organizations are not teachers, and Anti-Virus software vendors are not students. Malware analysts and researchers analyze threats every day without the guidance of testing organizations, and they see a lot more real-world threats than testing organizations.

 

18 hours ago, andone said:

In over 2000 years we still have the same 10 commandments as  "behavior standard" , unmodified, and the expectation is for us to behave accordingly .

At least that was my understanding about a behavior blocker; once you have a set of rules, should be good for life....

Technology changes rapidly, whereas humans do not. While it may be possible to regulate human behavior with a relatively short list of commandments/rules/etc. please note that regulating malware takes far more than that.

Share this post


Link to post
Share on other sites

Good Day @GT500

We would appreciate it if you would explanations about the following my ambiguities.

Except for  AV-C real world test March 2013, with 100% protection in the test, Emsisoft has not been able to detect all threats (100% blocked without user dependent) to this day. Has 6 years of real world testing and more than 50 monthly tests for Emsisoft been not a good time to improve and stable a protection rate?

 

After the 2017.7 updates, Emsisoft has not been able to get the full score (100% blocked without user dependent)! If in the tests preceding this update, the product was able to fully detection all threats with the user dependent (Behavior Blocker) like Feb 2017, March 2017 and Jan 2017, technically, after the 2017.7 updates, the detection rate must be 100% (blocked without user dependent) in the tests. What is the explanation for this issue?

Looking forward to receiving your comments.

Share this post


Link to post
Share on other sites
36 minutes ago, High-Momentum said:

Good Day @GT500

We would appreciate it if you would explanations about the following my ambiguities.

Except for  AV-C real world test March 2013, with 100% protection in the test, Emsisoft has not been able to detect all threats (100% blocked without user dependent) to this day. Has 6 years of real world testing and more than 50 monthly tests for Emsisoft been not a good time to improve and stable a protection rate?

 

After the 2017.7 updates, Emsisoft has not been able to get the full score (100% blocked without user dependent)! If in the tests preceding this update, the product was able to fully detection all threats with the user dependent (Behavior Blocker) like Feb 2017, March 2017 and Jan 2017, technically, after the 2017.7 updates, the detection rate must be 100% (blocked without user dependent) in the tests. What is the explanation for this issue?

Looking forward to receiving your comments.

We believe that a big configuration error was to leave the scan level in Default.

The best detection rate is only achieved by setting the scan level in THOROUGH

Share this post


Link to post
Share on other sites
2 hours ago, High-Momentum said:

Except for  AV-C real world test March 2013, with 100% protection in the test, Emsisoft has not been able to detect all threats (100% blocked without user dependent) to this day. Has 6 years of real world testing and more than 50 monthly tests for Emsisoft been not a good time to improve and stable a protection rate?

During most of that time, the Behavior Blocker was not automatic. How many of the tests would we have had a 100% in if the Behavior Blocker didn't ask the user before taking action?

 

 

1 hour ago, onbox said:

We believe that a big configuration error was to leave the scan level in Default.

The best detection rate is only achieved by setting the scan level in THOROUGH

I know that everyone probably sees me as just making excuses here, however please believe me when I say that the File Guard settings had nothing to do with the detection failures in the test.

Share this post


Link to post
Share on other sites

Thanks for response,

In the last 6 years, Emsisoft won 100% (blocked without user dependent) of the first test (March 2013), so why never did that happen again? Many other brands are able to score 100% at least a few times a year.

https://www.av-comparatives.org/tests/real-world-protection-test-march-june-2013/

In 2015, 2016, and 2017, Emsisoft has repeatedly been able to 100% by behavior Blocker and user dependent (Please check the following links) In version 2017.7, the Emsisoft behavior blocker defines everything by itself (First comes to Anti-Malware Network and blocks if it does not receive a response.), and thus the user does not interfere. With this update, your yellow section (user dependent) in the tests should be green (Blocked: Malware was successfully blocked by AV), but not yet.

https://www.av-comparatives.org/tests/real-world-protection-test-march-june-2015/

https://www.av-comparatives.org/tests/real-world-protection-test-february-june-2016/

https://www.av-comparatives.org/tests/real-world-protection-test-july-november-2016/

https://www.av-comparatives.org/tests/real-world-protection-test-february-june-2017/

Share this post


Link to post
Share on other sites
2 hours ago, High-Momentum said:

In 2015, 2016, and 2017, Emsisoft has repeatedly been able to 100% by behavior Blocker and user dependent (Please check the following links) In version 2017.7, the Emsisoft behavior blocker defines everything by itself (First comes to Anti-Malware Network and blocks if it does not receive a response.), and thus the user does not interfere. With this update, your yellow section (user dependent) in the tests should be green (Blocked: Malware was successfully blocked by AV), but not yet.

I'm not 100% certain what you're asking here. All of the 2017 tests at your link show user-dependent results for Emsisoft Anti-Malware, and in most of them we scored a 100% if you include the user-dependent test results.

Share this post


Link to post
Share on other sites

The test is a removal test. Apparently there were some threats that our cleaning engine had trouble removing after they had infected the system. Our developers will have to look into the data from AV-Comparatives to see what wasn't removed and why.

Share this post


Link to post
Share on other sites

Poor results in "performance test"
https://www.av-comparatives.org/tests/performance-test-october-2018/

The disappointing results are still in the "real world test"
https://www.av-comparatives.org/tests/real-world-protection-test-october-2018-factsheet/

Why is not the result in this test satisfactory?

 

The results of Emsisoft is not good at competitors.
The results of Emsisoft should be high and stable like Bitdefender, Avast and...

Share this post


Link to post
Share on other sites
4 hours ago, High-Momentum said:

Poor results in "performance test"
https://www.av-comparatives.org/tests/performance-test-october-2018/

The disappointing results are still in the "real world test"
https://www.av-comparatives.org/tests/real-world-protection-test-october-2018-factsheet/

Why is not the result in this test satisfactory?

 

The results of Emsisoft is not good at competitors.
The results of Emsisoft should be high and stable like Bitdefender, Avast and...

While Bitdefender works 100%, the Bitdefender engine in Emsisoft does not?

Share this post


Link to post
Share on other sites
4 hours ago, onbox said:

While Bitdefender works 100%, the Bitdefender engine in Emsisoft does not?

BitDefender's scan engine is supplemented by their own automated behavioral blocking technology. Since we only use their scan engine, our detections will always be different from theirs.

 

9 hours ago, High-Momentum said:

Why is not the result in this test satisfactory?

They tested on Windows 10, and Emsisoft Anti-Malware has an issue on Windows 10 where the first update can take an abnormally long period of time to initialize (it doesn't happen 100% of the time, but I expect that it must have happened in their test to have not achieved a perfect score).

As for the abysmal results for installation, I'm not entirely certain why that is. Possibly because we use an MSI installer, whereas I would believe that most of the other Anti-Virus software companies have built their own installers. MSI has its advantages, however it is rather slow. <- Sorry, I misread the column header in the results. It was installing applications, not installing the AV software. They tested Emsisoft Anti-Malware version 2018.9, which had some bugs in a new driver, so that's why the result was so poor. If the test was run with the current stable version (2018.10.1) then the results would be quite different. ;)

Share this post


Link to post
Share on other sites
On 11/15/2018 at 9:20 PM, GT500 said:

They tested on Windows 10, and Emsisoft Anti-Malware has an issue on Windows 10 where the first update can take an abnormally long period of time to initialize (it doesn't happen 100% of the time, but I expect that it must have happened in their test to have not achieved a perfect score).

Uh, any plans / ETA for a fix ? Speedy updates after the computer starts are very important IMHO...

You know, people tend to start checking their emails right after startup, opening all kinds of attachments
with viruses in them 😋

Thanks 🤗

Share this post


Link to post
Share on other sites
5 hours ago, Raynor said:

Uh, any plans / ETA for a fix ? Speedy updates after the computer starts are very important IMHO...

As far as I know the issue only happens immediately after installation. I don't think I've ever seen it happen again on the same computer, even after a restart of the system.

Share this post


Link to post
Share on other sites
Am ‎15‎.‎11‎.‎2018 um 21:20 schrieb GT500:

 They tested Emsisoft Anti-Malware version 2018.9, which had some bugs in a new driver, so that's why the result was so poor. If the test was run with the current stable version (2018.10.1) then the results would be quite different. ;)

So let´s wait for the Real-World Protection Test November 2018 and see the results. 8)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.