We’ve been thrilled with the enthusiastic embrace of PhishTank by an active community. Check those stats! Despite our unspoken office contest to submit and verify as many phishes as possible, all the OpenDNS employees are being blown off the Top Submitters/Verifiers lists (or soon will be) by active individuals around the Internet. That’s a good sign!
This is day five. We’ve been making adjustments and changes all week in response to comments and learnings. We’re not done, so keep telling us how to improve.
There are a lot of different questions we’ve fielded, and ideas we’ve heard. Here are some answers and comments and a quick look ahead on PhishTank.
Screenshots
We know that screenshots of suspected phish sites are valuable in judging a submitted URL, and help avoid visiting a potential phishing site (which should be done with care!). We also know that sometimes the screenshot doesn’t work very well. Please use the “Something wrong with this submission?” link on the right-hand side to alert us. We’ll add a specific choice for “Screenshot problem” shortly. The development team has a ticket for improving this key feature. It’s not a binary issue, but it will get better.
Duplicate URLs
There should not be any…but there are some as I type this. We know why this mistake happened, and it’s being fixed today. My apologies.
Wrong URL picked from email submissions
With some phish submissions via email, the PhishTank software chooses the wrong URL as the phish URL to judge. We’re working to improve our choice, of course. If we’ve got it wrong, please tell us via the “Something wrong with this submission?” link, rather than voting on an obviously biffed URL.
Redirects
Some phishing sites mask their final destination URL by using open redirect URLs at legitimate services. The final destination should certainly be marked as a phish, but the phish URL being judged is often the masked URL. Our take, for now, is that both the full original URL, including the redirect, and the final destination URL are phishing. The point? If someone can click on the URL and get to a phishing site, it’s bad news. This is an understandably grey area, and we’re happy to revisit as the data tells new stories.
Flags
Flags are what we call the notes appended to individual phish IDs via the “Something wrong with this submission?” link. These are read with interest, and help us as PhishTank administrators know where to focus our attention. Please continue to use them!
We are considering whether or not to make them visible to more than just administrators. They are informative, but wondering whether they will bias votes or not. PhishTank doesn’t tell you how others have voted on a submission until you vote because we hope you make your own judgment.
We’re undecided here. Thoughts on making these notes visible?
Judging a site that is offline
We’re continuing to tweak our code for judging (and re-checking) whether a submission is online or offline. We know it’s not 100% accurate, in part due to the normal volatility of phishing sites. If a site is offline, please do not vote. Instead, flag it for review via the “Something wrong with this submission?” link. We use these examples to test and improve our software for checking online status. Our belief is that it’s inappropriate to vote on a site that is not available. Of course, some URLs on their own show phishing intent and no possibility of mistakenly hurting legit folks if identified as phishes; there are grey areas. Help us work to define them further.
Making a mistake
I’ve made a few mistakes already where I mistakenly judged a submission as a phish (or “NOT a phish”) because my mouse finger was moving faster than my brain.
The good news? The community gets it right, and a single mistake vote won’t damage the overall judgment.
There is no need to notify us if you make a mistake. We’re not going to change individual votes. Your choices do matter: better choices will increase the “weight” of your future votes. Still, we’re also going to bake in a (small) allowance for this kind of mistake when judging an individual’s contributions.
We’re going to modify the two links (Is a phish / NOT a phish) to try and make them more distinct and less prone to mistakes.
Displaying suspected phish emails
Several people have asked why we don’t display the suspected phish emails, too. We do store the submitted email, and try to append extra information based on headers where possible. Viewing the email might help in making a better judgment, but there are two elements holding us up.
First, we’re concerned about usability. Before launch, some of the email information was displayed. The individual phish detail page was cluttered. We didn’t solve that problem before launch, but it is solvable.
Second, under no circumstances should PhishTank display personal information about the submitter. With email submissions, that requires extra care. Until we get it right, we will leave the source of the email (for example) behind the scenes.
We are considering screenshots of the emails, although the rendering in different email clients is notably more varied even than web browsers.
MTA (Mail Transfer Agent) information from the email is something we hope to break out, too, for display and API query.
In any event, we know the email itself has valuable information for PhishTank beyond just the phishing URL, and we’re thinking it through.
whois and ASN data
We are adding whois and ASN (Autonomous System Number) data to the submissions, although not currently displayed, primarily because the output of these two fields (especially whois) is so varied. We’ll figure it out.
Coming sooner, probably, are RSS feeds by ASN, so webhosts, ISPs and other organizations can subscribe to notifications about verified phishes on their networks. PhishTank doesn’t do takedowns, but certainly hopes that the data proves useful for those in a position to act.
RSS feeds
The focus for sharing information has been the API (check out the new diagram). But we believe in the simplicity of RSS feeds, too. Beyond the RSS feed for this blog, the site already offers individuals a personal feed to track their contributions. Find it on the My Account page.
We will offer more RSS feeds over time, like the ASN feeds noted above.
Text file of all verified phishes
The API does not offer a way to pull every single verified phish, purposefully. It would not be efficient for developers or PhishTank. However, we’ve heard many requests for a straightforward text file, updated frequently, that lists every verified phish.
We will offer such a file. Goal is to have this up and running sometime next week, barring other interruptions. Availability will be announced on this blog (http://www.phishtank.com/blog/) and in the API documentation.
More API calls coming
There’s more to come with the API. Most immediately, the API will offer calls to submit an email or URL to PhishTank, in addition to check them, as it does now. All that’s needed is some documentation. Stay tuned. If you want something else from the API, just ask. We’ll try to say yes to all reasonable requests; we don’t want to build applications, we want to enable application building.
A few people have written in asking about API limits. I’ll just quote the specific section of the FAQ:
There is no set usage limit. Extreme use will be noted, and we would ask that you contact PhishTank if you plan to use the API heavily. We welcome such usage, but would prefer to hear about it before it begins. PhishTank reserves the right to terminate API usage for accounts which abuse the free privilege.
As we learn more, we’ll get more specific.
Phew… more than enough for now. Comments invited and expected.
