VPN Audits Are Important, But They Don’t Paint a Full Picture – CNET [CNET]

Commentary: I’d take an audited VPN over an unaudited one, but let’s talk about what audits actually “prove.”

You put a ton of trust in your VPN provider to protect your privacy when you go online. A virtual private network encrypts your internet traffic while routing it through a secure server. In doing so, the VPN keeps your online activity hidden from your internet service provider, mobile carrier, network administrator, government and any other entity looking to snoop on what you’re doing on the internet. 

Without a VPN, your ISP has eyes on what websites you’re visiting and what apps you’re using. Your ISP collects information about your online activity and can share it with advertisers and law enforcement. When you use a VPN, you’re essentially swapping out your ISP with your VPN as the gatekeeper to your connection to the internet — so you need a VPN that won’t sell you out. 

The core promise of any good VPN is that it doesn’t collect or store logs of its users’ online activity. But how do you know if your VPN provider is actually doing what it promises? The truth is, you don’t — you just have to take the VPN provider’s word for it. In an effort to bolster trust, many VPN providers have begun undergoing third-party audits of their privacy policies and app security. 

VPN companies love to boast that successful audits “prove,” “validate,” “verify,” “confirm,” “certify” and “authenticate” their no-logs policies and app security. In reality, an external audit can only confirm the auditing team’s findings during the course of the audit itself (typically about a week or two). This means that you still have to take the VPN’s word for it for the other 50 weeks of the year — or more if the VPN doesn’t undergo an audit every year. 

Still, external audits are a crucial ingredient in a VPN’s overall stance on privacy and transparency. Here’s what you need to know about VPN audits, their limitations and what a VPN should be doing to gain your trust. 

What is a VPN audit?

A VPN audit calls on an independent accounting or cybersecurity firm to examine the company’s privacy policies and security infrastructure. There are two main types of audits that VPN companies generally commission: a privacy audit and a security audit. 

A VPN privacy audit is often completed by an accounting firm and looks into the VPN provider’s terms of service, privacy policy and no-logs policy to ensure that the VPN is indeed doing what it promises in those policies. (You’ll typically see privacy audits done by one of the “Big Four” accounting firms: Deloitte, KPMG, PwC and Ernst & Young). The audit will evaluate things like how the VPN provider handles user data, what data it collects, what data is saved on its servers, how long data is saved and for what purpose. 

The typical privacy audit also dives into whether the VPN provider collects usage and/or connection logs. While no VPN that truly cares about your privacy will log identifying data like your IP address, some aggregated connection logging is necessary for things like troubleshooting connection issues, fixing bugs, preventing abuse, diagnosing crashes, optimizing performance and enforcing simultaneous connection allowance. It’s impossible to operate a VPN service without collecting at least some connection logs, which can include data like connection timestamps, amount of data transferred while connected, server load (how many users are connected to a particular server), app diagnostic data and user IP address. 

When a VPN says that it’s a “no-logs” VPN provider, it typically means that it doesn’t collect any usage logs, meaning data related to your online activity, including the sites you visit, apps you use, your DNS requests and unencrypted communications. Any VPN collecting usage logs would undermine the entire premise of using the VPN in the first place. This is why it’s so important to ensure that the VPN you’re using is trustworthy and won’t log your usage data and potentially sell it to third parties — which is what many free VPNs may be doing. Also, depending on the VPN’s jurisdiction, local laws may obligate a VPN provider to share user data with authorities. The VPN you’re using shouldn’t have any data about you or your online activity that it would be able to share with authorities or any other third party.

A VPN security audit differs from a privacy audit in that it focuses on the VPN’s infrastructure rather than its policies and is typically handled by dedicated cybersecurity firms like Cure53, F-Secure or VerSprite. The VPN gives the auditing company access to its internal systems, and the security audit evaluates the security of the VPN’s software and infrastructure to look for potential vulnerabilities in the source code that could put users at risk. Some security audits focus on a VPN’s app for a single operating system or protocol. For instance, ExpressVPN commissioned separate security audits for each of its apps, along with its Lightway protocol and Aircove router. Other security audits take a more generalized approach to software and infrastructure security, like the audit NordVPN commissioned in 2022. 

Although a VPN doesn’t technically need to publish the results of its audits, the general practice is to publish at least a summary of the audit results. Here at CNET, we ideally would like VPN providers to publish their full audit reports and make them available to the general public in the interest of full transparency. Sometimes, restrictions imposed by the auditing company may prevent the VPN from publishing the full report publicly. However, the audit reports are typically made publicly available online via a link from the VPN provider’s website. A VPN audit report is a thorough documentation of the entire audit process, covering everything from the auditor’s methodology to the scope of the audit, vulnerabilities identified (ranked by severity), miscellaneous issues identified and recommendations. 

Why are VPN audits important?

A VPN company is under no obligation to undergo any sort of external audit. Commissioning an audit can be expensive and time-consuming, but VPN audits are important for several reasons that benefit both the VPN provider as well as the end user.

First, VPN audits help establish a crucial trust signal from the VPN provider that it’s not just blowing hot air when it says that its software and infrastructure are secure and that it collects no logs. This is especially important considering the level of trust you need to put into a company in an industry that’s notoriously opaque. However, it’s encouraging to see more and more VPNs hopping on the audit bandwagon and embracing a commitment to transparency. A VPN can say whatever it wants about its security and stance on no logging, but without an independent audit, it’s extremely difficult to give any amount of credence to those claims.

Similarly, external audits can help VPNs differentiate themselves from the competition. While an unaudited VPN isn’t necessarily a low-quality VPN that you should automatically distrust, an audited VPN naturally comes across as more trustworthy. If I’d personally have to choose between two similar VPNs, one audited and the other unaudited, I’d opt for the audited VPN every time. To me, it’s almost as if an unaudited VPN has something to hide. Of course, that may not be the case at all for many unaudited VPNs, but given the extreme level of trust I have to place in my VPN provider, I’d rather not take chances. An audit signals that a VPN provider is confident enough in the soundness of its privacy and security posture to allow professional auditing firms access to the VPN’s inner workings and report on their findings. Additionally, when a VPN company undergoes regular audits, is transparent enough to share its full audit reports with the public and exhibits a commitment to addressing potential vulnerabilities identified in the audits, I’ll put even more trust in that provider. An audit isn’t the be-all and end-all of VPN trustworthiness, but it’s still a major trust signal. 

VPN audits also help identify vulnerabilities in the VPN’s software or infrastructure and offer recommended fixes for those vulnerabilities, regardless of their severity. This helps beef up the VPN’s security and privacy protections and ultimately helps better protect you as the end user. 

VPN audit limitations

At CNET, we place a heavy emphasis on audits when evaluating a VPN’s overall privacy and transparency. However, VPN audits have their inherent limitations — the most prominent of which is that audits can only provide an assessment of a VPN’s privacy and security during a short window of time. You can only know if a VPN was secure and if it didn’t log during the duration of the audit itself, not before and not after. 

Even a seemingly innocuous app update following the completion of an audit could have potentially serious consequences for user privacy. Case in point: ExpressVPN’s Windows app underwent a successful audit in 2022, during which cybersecurity firm F-Secure “did not identify vulnerabilities which can be exploited to cause information disclosure, IP address leakage or [remote code execution] in the ExpressVPN Windows application.” However, shortly afterwards ExpressVPN issued an update to the Windows app that introduced a vulnerability that resulted in DNS leaks under certain conditions when the split tunneling feature was enabled. The vulnerability went unnoticed for years until I came across it during my testing and reported it to ExpressVPN. 

This is why it’s critical for VPNs to conduct external audits on a consistent basis. An audit here and there every few years is better than nothing, but a regular annual audit cadence can go a long way in boosting a VPN’s level of trustworthiness in addition to catching dangerous vulnerabilities that could potentially go unnoticed for years. 

Open-source VPN providers like Mullvad, Proton VPN and PIA are able to mitigate against this particular pitfall by making their source code available to the general public for scrutiny. This helps keep these VPNs honest while also allowing anyone with the technical chops to identify any potential vulnerabilities at any time — no need to wait for an official audit.

Mullvad is working on taking it to the next level by making its server infrastructure fully auditable by anyone who wishes to look into it at any time with its System Transparency initiative. Mullvad says on its website, “Achieving transparency on the server side is a … challenge, as merely open sourcing our server software is not enough. We want our users to be able to verify and audit what is currently running on the VPN server they are connected to.” 

Having continuously auditable servers will get you about as close as you can get to being able to actually verify a VPN’s privacy and security posture. Until then, the best you can do is to take your VPN’s word for it that it’s safe to use when it’s not being audited.

Other ways to ensure your privacy with a VPN

External audits are just one piece of the (complex) VPN puzzle, and an imperfect piece at that. Other than through an audit, a VPN provider can back up its no-logs claims if it’s subpoenaed in a legal case. A truly “no-logs” VPN should not have any information to provide law enforcement in these cases. Last year, Mullvad was involved in a case in which it was unable to supply user data to law enforcement, and PIA has had its no-logs claims tested in court on multiple occasions. If you want to know if a VPN is trustworthy, research its audit history as well as its involvement in any legal proceedings. 

Look also for VPN transparency reports that detail the number of subpoenas, court orders and warrants the VPN company was served during a given period of time and how the company responded to those requests. Transparency reports, like audits, can boost a VPN’s trustworthiness.

Ideally, for optimal privacy your VPN provider should be located in a privacy-friendly jurisdiction outside the reach of the 14-eyes data sharing alliance, like Panama or the British Virgin Islands. That said, if the VPN you’re using truly doesn’t log your activity, then it shouldn’t matter much. Other things to keep in mind with a VPN is whether it has a kill switch, DNS leak protection and a RAM-only server infrastructure, all of which can help ensure your privacy while connected to the VPN. 

It’s also always a good idea to peruse your VPN provider’s privacy policy to get an idea of how it handles your data. What data does it collect and for what purposes? What other entities does the provider share your data with if any, and under what circumstances? Does the VPN provider keep user data completely in-house or does it share it with its parent company and/or sibling companies (if applicable)? All of this information should be in a VPN’s privacy policy. And if it’s not, or if you’re at all uncomfortable with the level of data collection or sharing, look for a different provider.

It takes a lot for a VPN to be trustworthy. VPNs love to inflate their capabilities in marketing. But by doing your research, knowing what trust signals to look out for and understanding their limitations, you can get a pretty good idea of which VPN is actually doing what it says it’s doing — even if you can’t verify it with complete certainty.