How Mailvelope Does Analytics
Analytics is a sensitive topic for every person who makes the point of taking extra steps to protect their privacy online. Mailvelope’s mission has always been to provide everyone with an easy to use, secure and transparent tool to encrypt their email communications. Until now there was no analytics built-in to the product.
Of course these days, every organization will claim they need access to your data to “improve your experience”, and it’s often easier and safer to blanketly reject all requests. This is why we wanted to take the time today in this article to explain why this analytics project will be useful for the community as a whole and how such anonymity is built in order to be respectful of your privacy.
Why do we need analytics?
Currently the Mailvelope team only has access to built-in web extension marketplace analytics about installation and uninstallation. However this situation has created several problems. To put it simply, the Mailvelope team lacks some of the most basic data needed to understand and fix some of the biggest usability problems its users are facing.
In those marketplace analytics, we have seen a concerning pattern where some users install the extension but give up and uninstall within a few days. Unfortunately, we have no tools to know at which steps they are dropping out. Is it because they cannot generate a key? Is it because they do not understand how to send an encrypted email? Is that because they have no one to send an encrypted email to? Which webmail providers are they attempting to use Mailvelope with? These are some of the questions we need some answers to if we want to further our goals, and we feel analytics is the right tool for the job.
How do these analytics work?
Using Clean Insights, we aggregate and minimize data before it ever leaves your device. This means your device might report something like “A user generated key pair twice this week”, but would not include key generation timestamps or identity information. To prevent correlation of analytics requests with the underlying events being measured, analytics data is sent to the server well after its collection (once per day or once per week depending on what’s being measured). On the server, we use the Clean Insights Matomo Proxy to remove any identifying information before storing the data in Matomo.
We’re left with a simple (some might say clean) dataset that can’t be used to identify or track a particular user but can help us answer critical questions like "Which onboarding steps do users encounter before giving up?"
The underlying software packages, Matomo, Clean Insights’s JavaScript SDK, and Matomo Proxy are each open source and auditable, as is our implementation within Mailvelope.
How can I turn analytics on or off?
The extension will only collect analytics data with your consent. Mailvelope already lets you choose whether to share your public key on the Mailvelope keyserver. Similarly, we ask for your permission to collect and analyze some information before one bit of analytics data leaves your device. New users are presented with the choice to contribute analytics data upon install:
Any user can grant or revoke consent for analytics in the Options menu under "Analytics".
What will become of this data?
In the next few months, we'll share what we learn about webmail provider popularity with the community and we'll prioritize our development based on these new (clean) insights.
Credits
Thanks to all project contributors:
- John Hess from the Guardian Project for concept and developement
- Carrie Winfrey from Clean Insights for UX design
- Remy Bertot from Passbolt for communication
- With support from UXFund and OTF
See also the Mailvelope Case Study on the blog of the Guardian Project.
