Comments on: XML data file of online, valid phishes from PhishTank http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/ A blog about and from PhishTank, a collaborative clearinghouse for data about phishing. Thu, 21 Aug 2008 06:37:08 +0000 http://wordpress.org/?v=2.0.4 by: John Nagle http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47819 Sun, 14 Oct 2007 04:54:53 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47819 The XML file started containing useful data again on Friday, October 12th. Thanks. Incidentally, it would help if the file was updated as an atomic operation. Occasionally, we see a partially written file, if we happen to read it while it's being rewritten. We have to read the file twice at 30 second intervals and compare, rereading until we get the same contents twice in a row. It would be better to write a new file on each update, then move or link it to the name of the distributed file. The XML file started containing useful data again on Friday, October 12th. Thanks.

Incidentally, it would help if the file was updated as an atomic operation. Occasionally, we see a partially written file, if we happen to read it while it’s being rewritten. We have to read the file twice at 30 second intervals and compare, rereading until we get the same contents twice in a row. It would be better to write a new file on each update, then move or link it to the name of the distributed file.

]]> by: John Nagle http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47487 Fri, 12 Oct 2007 04:56:35 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47487 Something has gone very wrong with the XML file of PhishTank data at "http://data.phishtank.com/data/online-valid". Today, it reads: − 2007-10-12T04:30:01+00:00 0 That's the entire file. Valid XML, no entries. Something is very broken. Something has gone very wrong with the XML file of PhishTank data at “http://data.phishtank.com/data/online-valid”. Today, it reads:

−
2007-10-12T04:30:01+00:00
0

That’s the entire file. Valid XML, no entries. Something is very broken.

]]> by: Tom http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47408 Thu, 11 Oct 2007 10:58:15 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-47408 http://data.phishtank.com/data/online-valid/index.xml has been returning 0 entries since yesterday some time. Please advise http://data.phishtank.com/data/online-valid/index.xml has been returning 0 entries since yesterday some time.

Please advise

]]> by: Steve Garvey http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-42530 Fri, 14 Sep 2007 21:13:25 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-42530 The file is NOT sent compressed to my gzip capable browser (Firefox) Me: Accept-Encoding: gzip, deflate Your server: Content-Length: 6584660 What if someone writes a script to retrieve it without an "Accept-Encoding" header? As stated previously, you should be serving a gzip'd file. The file is NOT sent compressed to my gzip capable browser (Firefox)

Me: Accept-Encoding: gzip, deflate

Your server: Content-Length: 6584660

What if someone writes a script to retrieve it without an “Accept-Encoding” header? As stated previously, you should be serving a gzip’d file.

]]> by: John Roberts http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-9216 Mon, 05 Mar 2007 18:09:26 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-9216 Andres, the file is transparently gzipped across the wire as long as the client requesting the file support gzip. Andres, the file is transparently gzipped across the wire as long as the client requesting the file support gzip.

]]> by: Andres Riancho http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-9069 Sun, 04 Mar 2007 15:39:24 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-9069 Great work, the database is really amazing! Only one simple question, why isnt the xml file compressed ? dz0@sock3t:/tmp$ du -sh index.xml 6.4M index.xml dz0@sock3t:/tmp$ bzip2 index.xml dz0@sock3t:/tmp$ du -sh index.xml.bz2 292K index.xml.bz2 dz0@sock3t:/tmp$ The size reduction is amazing, you would save a lot of bandwidth ! Maybe the compression algorithm should be "zip" instead of bzip2 to make it easy to decompress in all programming languages and operating systems. -- Andres Riancho Great work, the database is really amazing! Only one simple question, why isnt the xml file compressed ?

dz0@sock3t:/tmp$ du -sh index.xml
6.4M index.xml
dz0@sock3t:/tmp$ bzip2 index.xml
dz0@sock3t:/tmp$ du -sh index.xml.bz2
292K index.xml.bz2
dz0@sock3t:/tmp$

The size reduction is amazing, you would save a lot of bandwidth !
Maybe the compression algorithm should be “zip” instead of bzip2 to make it easy to decompress in all programming languages and operating systems.

–
Andres Riancho

]]> by: phishthis http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-399 Tue, 05 Dec 2006 19:52:35 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-399 Thank You OpenDNS for having this data. Hopefully, as one who finds phishing and pharming as totally repulsive, this will help the yet-unwashed of the internet to learn what phishing is, and perhaps our efforts will keep them safer and spare them the time in repairing their good name(s). Mark Founder, President, and "Bottle Warsher" of the London Antiphishing Society, near Arkansas Nuclear One. Thank You OpenDNS for having this data. Hopefully, as one who finds phishing and pharming as totally repulsive, this will help the yet-unwashed of the internet to learn what phishing is, and perhaps our efforts will keep them safer and spare them the time in repairing their good name(s).

Mark
Founder, President, and “Bottle Warsher” of the
London Antiphishing Society,
near Arkansas Nuclear One.

]]> by: Ian http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-225 Wed, 15 Nov 2006 09:42:52 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-225 Perhaps you ought to make the server serve the RSS feed as static data, and kick out a 304 header if it is not modified? That should help bandwidth-wise. Perhaps you ought to make the server serve the RSS feed as static data, and kick out a 304 header if it is not modified? That should help bandwidth-wise.

]]> by: MASA http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-223 Wed, 15 Nov 2006 06:04:33 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-223 It's out, http://phishtank.com/sitechecker (redirects to the extension's homepage) It’s out,

http://phishtank.com/sitechecker (redirects to the extension’s homepage)

]]> by: MASA http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-109 Fri, 20 Oct 2006 02:52:47 +0000 http://www.phishtank.com/blog/2006/10/17/xml-data-file-of-online-valid-phishes-from-phishtank/#comment-109 I am using this data for a firefox extension that protects the user from phishpages based on data from phishtank. The extension is in the translation stage right now. I am using this data for a firefox extension that protects the user from phishpages based on data from phishtank.

The extension is in the translation stage right now.

]]>