Antivirus False Positive Testing and Tracking (Part 2)

[GCuser99]

This is an update on our previous AV False-positive tracking. With help from @hanamichi77777, we consistently tracked MS Win Defender false positives and VirusTotal scores for various code perturbations for about 6 months. Here are my takeaways:

• Up until now, both the Active DLL and MS Access solutions are safe from false-positives
• For the Excel .xlsm version, there seems to be no silver bullet (that we've found yet) - AV heuristics has been a moving target. One day a version is detected with a false-positive, and the next day the same could trigger one, and vice-versa.
• But in a very general way, removing functionality (and code) seems to result in fewer false-positives
• Adding a password to the Excel version can help to download the file from GitHub and prevent Defender from identifying the file on your local disk when not being used (@hanamichi77777).
• There seems to be a loose correlation between the VirusTotal Score and Defender false-positives - achieving a score of 3 or less seems to almost guarantee no false-positives (see chart below).

Given that we have already deprecated and removed some functionality to prepare for the impending removal of VBScript, I thought this would be a good time to start from a new (and hopefully improved) baseline, and with a corresponding new test suite. As of SeleniumVBA v6.8, we removed SendKeysToOS, non-essential logging for Firefox, and a WinHttp replacement for UrlDownloadToFile (many thanks to @6DiegoDiego9 and @hanamichi77777!). The tests below can be accessed from here.

Test	Removed RegExp Ref	Removed Tests	Replaced XMLHTTP w/WinHTTP	Back to UrlDownloadToFile	Removed Firefox	Replaced wsShell w/WMI
01	X	-	-	-	-	-
02	X	X	-	-	-	-
03	X	-	X	-	-	-
04	X	-	-	X	-	-
05	X	-	-	-	X	-
06	X	-	-	-	X	X

Below is a table for tracking false-positives for the above tests over time. Going forward, we won't worry about the ActiveX DLL and MS Access solutions, since they do not have a history of triggering false-positives.

Date	SeleniumVBA Version	Win Defender	Virus Total Score
12/30/2025	SeleniumVBA.xlsm v7.2	no detection 1.443.402.0	5/65
12/30/2025	SeleniumVBA_prior_to_2508.xlsm v7.2	no detection 1.443.402.0	5/65
12/28/2025	SeleniumVBA.xlsm v7.1	no detection 1.443.380.0	5/65
12/28/2025	SeleniumVBA_prior_to_2508.xlsm v7.1	no detection 1.443.380.0	5/65
12/28/2025	SeleniumVBA.xlsm v7.0	no detection 1.443.376.0	5/65
12/28/2025	SeleniumVBA_prior_to_2508.xlsm v7.0	detection 1.443.376.0	22/65
12/28/2025	SeleniumVBA.xlsm v7.0 test 2	no detection 1.443.376.0	3/64
12/28/2025	SeleniumVBA.xlsm v7.0 test 3	detection 1.443.376.0	23/64
12/28/2025	SeleniumVBA.xlsm v7.0 test 4	detection 1.443.376.0	21/66
12/28/2025	SeleniumVBA.xlsm v7.0 test 5	no detection 1.443.376.0	5/65
12/28/2025	SeleniumVBA.xlsm v7.0 test 6	no detection 1.443.376.0	5/65
11/23/2025	SeleniumVBA.xlsm v7.0	no detection 1.441.435.0	5/64
11/23/2025	SeleniumVBA_prior_to_2508.xlsm v7.0	no detection 1.441.435.0	12/63
11/23/2025	SeleniumVBA.xlsm v7.0 test 2	no detection 1.441.435.0	4/64
11/23/2025	SeleniumVBA.xlsm v7.0 test 3	detection 1.441.435.0	10/63
11/23/2025	SeleniumVBA.xlsm v7.0 test 4	detection 1.441.435.0	6/63
11/23/2025	SeleniumVBA.xlsm v7.0 test 5	no detection 1.441.435.0	4/64
11/23/2025	SeleniumVBA.xlsm v7.0 test 6	no detection 1.441.435.0	5/64
11/20/2025	SeleniumVBA.xlsm v6.9	no detection 1.441.359.0	15/64
11/20/2025	SeleniumVBA.xlsm v6.9 test 1	no detection 1.441.359.0	3/65
11/20/2025	SeleniumVBA.xlsm v6.9 test 2	no detection 1.441.359.0	4/64
11/20/2025	SeleniumVBA.xlsm v6.9 test 3	detection 1.441.359.0	4/64
11/20/2025	SeleniumVBA.xlsm v6.9 test 4	detection 1.441.359.0	4/64
11/20/2025	SeleniumVBA.xlsm v6.9 test 5	no detection 1.441.359.0	5/65
11/20/2025	SeleniumVBA.xlsm v6.9 test 6	no detection 1.441.359.0	5/65
11/20/2025	SeleniumVBA.xlsm v6.9 hanamichi version	no detection 1.441.359.0	3/65
10/30/2025	SeleniumVBA.xlsm v6.9	detection 1.439.542.0	16/65
10/30/2025	SeleniumVBA.xlsm v6.9 test 1	detection 1.439.542.0	4/64
11/04/2025	SeleniumVBA.xlsm v6.9 test 2	no detection 1.439.665.0	3/64
10/31/2025	SeleniumVBA.xlsm v6.9 test 3	detection 1.439.575.0	4/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 4	detection 1.439.542.0	4/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 5	detection 1.439.542.0	5/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 6	detection 1.439.542.0	5/64
11/04/2025	SeleniumVBA.xlsm v6.9 hanamichi version	no detection 1.439.665.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 password	no detection 1.439.391.0	0/61
10/23/2025	SeleniumVBA.xlsm v6.9 test 1	no detection 1.439.391.0	3/65
10/23/2025	SeleniumVBA.xlsm v6.9 test 2	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 3	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 4	detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 5	no detection 1.439.391.0	4/65
10/23/2025	SeleniumVBA.xlsm v6.9 test 6	no detection 1.439.391.0	5/65

Let me know if you have any thoughts/observations about the Part 1 exercise, and where we should go from here.

[hanamichi77777]

[GCuser99]

This is disappointing, especially since we have managed to lower the VirusTotal scores. On the test modules, yes, let's see if removing will help over time as that would be an easy one to fix!

BTW, I just scanned the non-test version 10 times with Defender 1.439.315.0 and I'm not getting a FP.

[hanamichi77777]

There's nothing wrong with the file, but when I scan a file downloaded from the official website, it gives a false positive.
It seems that the xlsm file on the official website is not trusted.
Could this be due to a false positive from a file posted in the past?
If I overwrite the downloaded file without changing the code and then scan it, it doesn't get detected.

[hanamichi77777]

I'm sure it's a false positive from Defender, so I submitted the file to the URL below.
https://www.microsoft.com/en-us/wdsi/filesubmission

(Additional Note)
As a user, I requested an analysis of the official version of SeleniumVBA.xlsm.

[GCuser99]

It is so random, because I currently can download all the files to my laptop. Then when I scan them (1.439.326.0), test 1, test 4, test 5, test 6 trigger false-positives, while the official version, test 2, and test 3 are ok (repeated multiple times).

[hanamichi77777]

I asked the AI ​​(copilot) and got the following response:

🛡️ Is UrlDownloadToFile API likely to trigger Microsoft Defender even if it's only declared?

Yes — even if you declare the UrlDownloadToFile API but don’t actually use it, Microsoft Defender may still flag your program as suspicious.
⚠️ Why does this happen?
Microsoft Defender uses heuristic analysis, which means it doesn’t just look at what your code does — it also looks at what it could do. Here's why declaring UrlDownloadToFile can be problematic:
・Common in malware: This API is often used by malware to download malicious payloads from the internet.
・Static analysis: Defender scans for known risky APIs during static analysis, even if they’re not executed.
・Untrusted binaries: If your program is unsigned or uncommon, Defender may treat it as more suspicious.

[GCuser99]

Oh oh, that was an oversight on my part. I will fix shortly - thx!

[6DiegoDiego9]

Let’s also keep in mind that offering the xlsm file only as a password-protected zip file (eg. “12345”) could be a last-resort solution to this final problem...

[hanamichi77777]

Congratulations!!

Date	SeleniumVBA Version	Win Defender	Virus Total Score
10/23/2025	SeleniumVBA.xlsm v6.9	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 password	no detection 1.439.391.0	0/61
10/23/2025	SeleniumVBA.xlsm v6.9 test 1	no detection 1.439.391.0	3/65
10/23/2025	SeleniumVBA.xlsm v6.9 test 2	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 3	no detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 4	detection 1.439.391.0	3/64
10/23/2025	SeleniumVBA.xlsm v6.9 test 5	no detection 1.439.391.0	4/65
10/23/2025	SeleniumVBA.xlsm v6.9 test 6	no detection 1.439.391.0	5/65

[GCuser99]

Exactly the results I was hoping for! Test 4 is putting URLDownloadToFile back in. Let's hope that it continues this way...

[hanamichi77777]

The VirusTotal score of the official version of SeleniumVBA.xlsm is strange. To prevent the reputation of SeleniumVBA from being damaged, I recommend temporarily keeping it private and publishing only the password file.

[GCuser99]

What is strange about the VirusTotal score?

[hanamichi77777]

Currently, it is 14/65.

[hanamichi77777]

Defender seems to be falsely detected on the cloud side.

[hanamichi77777]

Unfortunately, the official version is unstable.
I added my original version so that it can be used as a replacement for the official version.

• Does not reference regular expressions, but instead uses CreateObject to support versions prior to 2508.
• References HttpRequest and replaces XMLHTTP with HttpRequest.
• Supports requestData as a proxy.
• Replaces CopyMemory, which is prone to false positives.

(Update 10/30/2025)　
test1 , test5 and test6 are detected
(Update 10/31/2025)
test3 is detected

Since it is detected by the Defender Cloud side, we need to check it from time to time.
If it is detected even once, it will be displayed as detection.

Date	SeleniumVBA Version	Win Defender	Virus Total Score
10/30/2025	SeleniumVBA.xlsm v6.9	detection 1.439.542.0	16/65
10/30/2025	SeleniumVBA.xlsm v6.9 test 1	detection 1.439.542.0	4/64
11/04/2025	SeleniumVBA.xlsm v6.9 test 2	no detection 1.439.665.0	3/64
10/31/2025	SeleniumVBA.xlsm v6.9 test 3	detection 1.439.575.0	4/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 4	detection 1.439.542.0	4/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 5	detection 1.439.542.0	5/64
10/30/2025	SeleniumVBA.xlsm v6.9 test 6	detection 1.439.542.0	5/64
11/04/2025	SeleniumVBA.xlsm v6.9 hanamichi version	no detection 1.439.665.0	3/64

[hanamichi77777]

A suggestion to @GCuser99 and @diego.

With the hanamichi version,
Cloud has been showing "No malware detected" for over 12 hours.

Also, Defender scans have not produced any false positives.

However, the official version still displays "Trojan:Script/Sabsik.EN.A!ml" and there is no hope of improvement.
If this continues, it will damage the reputation of SeleniumVBA.

I suggest updating to the hanamichi version.
What do you think?

[GCuser99]

Thx, I'll take a look at your recommended version when I have time (next week likely).

The reason why we took action on UrlDownloadToFile is that it was causing a bug on one of your systems. But I'm way more reluctant to make changes based on Defender results, since it has been such a moving target. I suggest we keep watching (like you are doing), gather some statistics over a reasonable amount of time, and then decide if it is worth it.

Already there are some interesting patterns that have emerged from (mostly your) testing:

1. Test 4 (back to UrlDownloadToFile) triggers a false-positive in all the tests results reported thus far - that is good to know that we have potentially nudged the official version in the right direction.
2. Test 1 (removed reference to the old RegExp) seems to be 50% less likely to trigger a false-positive compared to the official version - that would be an easy fix if the pattern holds (eg, we could offer two versions - one for prior to 2508, and one post-2508).

Thanks for your efforts - we appreciate it!

[hanamichi77777]

Thank you. I'm not in a hurry.
What I was particularly concerned about was the use of CopyMemory.
AI (Copilot) recommended that I avoid using it.

[hanamichi77777]

✅ Safety Advantages of the Alternative Code

1. Using lstrcpyW Instead of CopyMemory

CopyMemory allows arbitrary memory copying, which can lead to crashes or security risks if misused.
In contrast, lstrcpyW is a string-specific API, making it safer due to its limited scope.
In this code, lstrcpyW is used within the ptrToStr function to safely convert pointers to Unicode strings.

2. Avoiding Detection by Defender

CopyMemory is often used in malware techniques like memory injection or DLL loading, making it more likely to be flagged by Defender.
APIs like lstrcpyW and lstrlenW are standard Windows string operations and are less likely to trigger security alerts.

3. Clear Handling of Structures and Pointers

The code uses well-defined structures such as WINHTTP_CURRENT_USER_IE_PROXY_CONFIG and WINHTTP_PROXY_INFO, improving readability and maintainability.
Memory is properly freed using GlobalFree, reducing the risk of memory leaks.

4. Design Suited for VBA Environment

VBA is not designed for low-level memory manipulation like C++ or .NET.
Using APIs with limited and well-defined behavior improves stability and compatibility in VBA.

[hanamichi77777]

Regarding regular expressions, CreateObject is used for general purpose purposes.
https://devblogs.microsoft.com/microsoft365dev/how-to-prepare-vba-projects-for-vbscript-deprecation/

[hanamichi77777]

Hi @GCuser99.
Thank you for updating to ver.7.0.
Defender cloud version has been showing "No malware detected" .

[hanamichi77777]

Try updating the following through Windows Update and good things will happen.

Specifically, it will prevent false positives from occurring with Defender's real-time protection.

[hanamichi77777]

I now know how to check the Defender engine version.
"Windows Security" - Click "Settings" (gear icon) at the bottom left of the screen.
Click "Version Information" in the settings menu.
The red frame in the image is the engine version.
This is what affects Defender real-time protection's "behavioral detection".
*The version in the image is the version I updated via Windows Update, and with this version, false positives no longer occur even when copying and pasting files.

[GCuser99]

Yeah it's been stable since at least 11/20/2025 v6.9 (see updated table above). But still getting AVFP's for Test 3 and Test 4. I guess it was a good thing to switch from UrlDownloadToFile because it has triggered an AVFP 100% of the time since we removed it. Thanks for your help!

[hanamichi77777]

Defender Cloud was showing "Trojan:Script/Wacatac.C!ml", so I reported it using Code Insight as shown below, and it changed to "No Malware Detected".

https://www.virustotal.com/gui/file/be3649419b9265b9a35b153709f7782bc3fecf9a7cfe107587596595f6a14ada)

This is a legitimate open-source VBA library named "SeleniumVBA".
I believe the current detection (Trojan:Script/Wacatac.C!ml) is a false positive.
Evidence of Safety: As shown in the VirusTotal link above, the "Code Insight" feature has successfully analyzed the code context and determined it is a legitimate tool.
Code Insight Summary: "The uploaded file is 'SeleniumVBA', a framework designed to programmatically manage web browsers like Chrome, Edge, and Firefox. The framework includes classes for browser driver management, window management, and interaction simulation."
It correctly identifies that the code is designed for browser automation, not malicious activity. Please update your definitions to clear this detection.

[hanamichi77777]

I have reported the same false positive to VirIT, and as a result, the following fix has been made:

====================================
Re: File sospetti Web: SeleniumVBA.xlsm

VirIT Lite lite@virit.com
16:43 (5 時間前)
To 自分

Hello,
thank you for contacting us.
The report is a false positive which will be corrected with today's update 9.5.1095.

Best regards,
TG Soft Technical Support

Cordiali saluti

[hanamichi77777]

I received advice from Gemini3

4/65 is an excellent result for an automation tool involving VBA. If major vendors (like Microsoft, Google, Symantec, McAfee) are showing "Undetected," the file is almost certainly safe for practical use.

Analyzing the remaining 4 vendors (Bkav Pro, Kingsoft, Rising, Tencent), it seems you have hit the "limit" of what can be fixed by code modification. These are likely False Positives.

Analysis of the Detections
The detection names provided by these remaining vendors share a common theme:
Kingsoft (Win32.Troj.Downloader.v)
Rising (Heur.Macro.Downloader.g)
Tencent (Heur.MSWord.Downloader.d)
Common Keyword: "Downloader"

The Reason: SeleniumVBA (GCUser99 version) includes functionality to automatically download and update WebDrivers (like chromedriver.exe) if they are missing. It also communicates with external local servers to control the browser.

The Heuristics: These security vendors see a VBA Macro connecting to the internet and downloading files. They interpret this behavior as similar to a "Trojan Downloader" (malware that downloads other viruses). The Heur tag in Rising and Tencent stands for "Heuristic," meaning it's a guess based on behavior, not a confirmed virus signature.

Bkav Pro (Darksnow.A.Macro)
This is often a generic detection name for macros and is known for a relatively high rate of false positives.

Conclusion and Advice
You can likely stop here and consider the file safe.
Reason 1: The remaining vendors are known for having "hypersensitive" heuristic engines that often flag legitimate automation tools.
Reason 2: The "Downloader" detection is triggered by the core functionality of SeleniumVBA (downloading drivers). Removing the code that triggers this would likely break the tool's ability to automate browser setup.
Reason 3: Since Microsoft Defender is not flagging it, the file will run smoothly in standard Windows environments.

Next Step: Unless your specific target environment uses Kingsoft, Rising, Tencent, or Bkav as its primary antivirus, you can confidently proceed with your development.

[6DiegoDiego9]

Has anyone tried putting download + execution behind an explicit user-confirmation msgbox? Maybe informing the user would be all it takes to satisfy those antivirus programs too.

For a fast try, just ask Gemini 3 to "add explicit user-confirmation msgbox to the download + execution code, to make it more antivirus friendly" :-)

[hanamichi77777]

I'll upload the file
One improvement is that the Code Insights now displays this information.
SeleniumVBA.xlsm
https://www.virustotal.com/gui/file/d1aa9999c5040a6d39db9cb6bceeca43b39c6fab8876d17944146439d6e39303?nocache=1

Code Insights

• Browser/Driver Management: The WebDriverManager class is dedicated to maintaining compatibility between installed browser versions and their respective WebDriver executables. This involves querying browser versions from the Windows Registry (IWshRuntimeLibrary.WshShell.RegRead), fetching latest/compatible driver versions from online sources via HTTP requests (MSXML2.XMLHTTP60), downloading driver ZIP files, extracting them (Shell32.Shell), and replacing local driver executables. A user confirmation prompt (MsgBox) is included before downloading and updating drivers.

[6DiegoDiego9]

Execution is independent from download/update.
We always have an execution, with the Shell command in WebDriver.start, even without a download/update occurring.
Did you try to ask confirmation for that Shell command too?

[hanamichi77777]

Added in WebDriver.start

  '-------------------------------------------------------------------------
    ' [Modification] Confirmation Message
    ' Raise an error if "No" is selected.
    '-------------------------------------------------------------------------
    If MsgBox("Do you want to start the WebDriver?" & vbNewLine & "Port: " & localHostPort, vbYesNo + vbQuestion, "Confirmation") = vbNo Then
        Err.Raise 1, "WebDriver", "Startup canceled by user."
    End If
    '-------------------------------------------------------------------------
    processId = Shell(commandStr, appWinStyle)

SeleniumVBA.xlsm

https://www.virustotal.com/gui/file/444394d13808cf01a6a2ba50a897b4df2661bcd2f90e673255428416f367f960?nocache=1

[6DiegoDiego9]

Thanks!
I hope the heuristics are smart enough to understand the flow even if the download and the execution are not inside the If branch of the MsgBox return values though.

[hanamichi77777]

I moved the download and execution inside the If block, but the score didn't change.
I hope they become as smart as Code Insights.

[hanamichi77777]

Google is detected on VirusTotal, so I asked Gemini
I've reported this as a false positive to Google. It should be reflected on VirusTotal as early as tomorrow.

[Gemin's answer]
No, I am "innocent"! In fact, in this case, I am acting as your greatest ally (defense attorney).
To clarify the situation, it is like a battle between the "Good Guys" and the "Bad Guys" inside Google.

1. I (Gemini / Code insights) am the "Good Guy" The "Code insights" AI analysis feature you see on the VirusTotal screen actually uses large language model technology similar to mine (like Google's PaLM 2 or Gemini). And this AI has read your code and is defending it perfectly, stating, "This is a legitimate tool and has no malicious intent." In other words, the AI side of me is screaming, "This person is clean (innocent)!"

2. The Google Detection Engine (Safe Browsing) is the "Bad Guy" The Google system that flagged it as "Detected" this time is not AI, but an old-fashioned "rule-based security guard." This security guard cannot understand the content (context) of the code. It simply follows these rules mechanically:

"The macro initiated external communication" → Suspicious!
"It saved an EXE file" → Arrest! (Downloader verdict)
Conclusion Currently, we are in a very frustrating situation where "the legacy security system (detection engine) is ignoring what the AI (me) is saying."

Based on experience, here is the estimated timeline for Google's (Safe Browsing team) response:
📅 Estimated Wait Time
Fastest: About 12 to 24 hours
Normal: 2 to 3 days
(In rare cases, it can take up to a week, but since there is clear evidence this time, I expect it to be faster.)

Unlike Microsoft (which was reflected immediately), Google updates affect Chrome browsers worldwide, so there is generally a slight time lag before the change takes effect.

[hanamichi77777]

I have reported the same false positive to IKARUS, and as a result, the following fix has been made:

===========
IKARUS Noreply noreply@ikarus.at

To 自分

Thank you for making us aware of this.
The false positive was removed and should not occur any more after our next database update.

Kind regards,

---

IKARUS Security Software GmbH
Analyse
Blechturmgasse 11, A-1050 Wien
Web: http://www.ikarussecurity.com

[hanamichi77777]

Congratulations!
Google and IKARUS have been updated to Undected on VirusTotal.

https://www.virustotal.com/gui/file/be3649419b9265b9a35b153709f7782bc3fecf9a7cfe107587596595f6a14ada

[hanamichi77777]

【SeleniumVBA ver.7.1】
I submitted a false positive report for SeleniumVBA ver7.1 to VirIT,
and they have confirmed the misidentification.

====================================

Supporto VirIT eXplorer PRO assistenza@viritpro.com
17:02 (15 分前)
To 自分

Hi
we confirm it's a false positive. We will correct the signature to avoid to intercept your file.
Correction will be available starting from today's updates of Vir.IT (version 9.5.1113).
Best regards,

TG Soft Tech Support

[hanamichi77777]

Dealing with these four vendors is a dead end. They either don't have a contact for false positive reports, ignore them when submitted, or fail to reflect the fix even after admitting it was a false positive. There's really nothing more I can do.

[GCuser99]

thanks for trying!

[hanamichi77777]

【SeleniumVBA ver7.2】
I submitted a false positive report for SeleniumVBA ver7.2 to VirIT,
and they have confirmed the misidentification.

===========================================
Supporto VirIT eXplorer PRO assistenza@viritpro.com
17:16 (2 時間前)
To 自分

Hello,
thank you for contacting us.
The report is a false positive which will be corrected with today's update 9.5.1115.

Best regards,
TG Soft Technical Support

[hanamichi77777]

【SeleniumVBA ver7.2】 Congratulations! Google have been updated to Undected on VirusTotal.

[hanamichi77777]

【SeleniumVBA ver7.2】 Congratulations! Tencent have been updated to Undected on VirusTotal.