Cyveillance IP list updated

2010-01-26 11:53:28 by chort

A while back I noticed Cyveillance, Inc were aggressively spidering my site. I found quite a few other references on the web to their anti-social behavior, including links to the recording industry's heavy-handed and borderline illegal tactics. In order to block them from my network, I compiled a list of their IPs.

It's been some time since I've actively monitored my firewall and over time the list had grown stale. I'd also previously been stymied on doing more research by my inability to figure out the nuances of some RWHOIS systems. Happily I made a breakthrough this week and I've been able to update my list, which I'll share for the good of humanity. The link above has the same list.

# Cyveillance @ Cogent

# Cyveillance @ Verizon (incomplete?)

# Previous(?) Cyveillance IPs

I'll try to update the text file over time to match current reality as best I can, but this blog post will go stale. I'm only putting the IPs here for spiders to find. If you want to use the list on your firewall, download the linked version. The list is admittedly incomplete since I haven't been able to reliably query Verizon for IPs (let alone other possible providers).

Updated 2010-03-28 to add, which came to me via a comment. Thanks for the tip!

Handy WHOIS tip

2010-01-26 08:48:31 by chort

While doing some research last night I finally figured out how to query a WHOIS server for all netblocks owned by a particular organization. For example, to find all netblocks owned by OrgID: NOC, do the following:

$ whois -a '> o !NOC'

In this case I'm using BSD whois, so the '-a' means "search ARIN". The other options are for the server. ARIN's WHOIS server interprets '>' as "show subordinate entries", the 'o' as "query for organizations", and the '!' as "search for handle or ID".

You should get output that starts like:

Resources Used By Organization:
Network Operations Center Inc. (AS21788) NOC 21788
[additional lines removed]

Linux users will need to adjust the flags passed to whois.

You can often get help from a specific WHOIS server by querying for '?'. This needs to be protected from your shell, so either backslash escape it, or wrap it in single-quotes. To get help from ARIN's WHOIS server do this:

$ whois -a \?

Final note: BSD whois doesn't appear to have a flag to force the RWHOIS protocol and different OSs have widely different ideas of what WHOIS ports are "well-known". For instance, OpenBSD has WHOIS and nothing else, while OS X has WHOIS++ and RWHOIS, but not WHOIS. FYI these are the ports:

whois           43/tcp          nicname
whois++          63/udp     # whois++
whois++          63/tcp     # whois++
rwhois          4321/udp    # Remote Who Is
rwhois          4321/tcp    # Remote Who Is

You can specify the port with the '-p' flag on BSD whois.

Blogs attract PHP scans

2010-01-24 23:54:49 by chort

I've been noticing that since I put up this blog I've been getting scans for common PHP files/site layouts. This is interesting because my main site hasn't been scanned for them at all during the same time period.

I also noticed that the majority of the spider traffic to my blog is from Baidu, in contrast with the rest of my site.

I had forgotten how fun it is to scan my webserver logs for patterns.

Is mounting VMDK files really that hard?

2010-01-19 22:50:06 by chort

One of my current projects at work is to create a pre-packaged virtual appliance that potential customers can install in their VMware virtualization environment to benchmark host performance and report it back to us. The data is used to make sizing and resource allocation recommendations for virtual deployments of our product. The issue I'm stuck on is reporting the data.

Obviously the preferred method would be a phone-home capability that simply ships the data directly from the VM to one of our servers, without the end-user having to do anything. The problem is that a lot of network operators (wisely) block outgoing connections by default. This is compounded by the fact that the appliance automatically gives itself an IP address via DHCP (to make installation easier), which means firewall exceptions are a non-starter.

Since phoning home via SMTP or HTTP probably won't even hit 70% success rate, I decided to not bother wasting time on those. The next idea was to write to a virtual floppy device, which is saved in the datastore as a .FLP file and could easily be downloaded by the end-user and e-mailed to us. A far-fetched idea (thought of by myself and another engineer on my team completely independently) is to use specially formatted DNS queries-- la Dan Kaminsky--to feed base64 encoded data to our server (since DNS queries are much more likely to be allowed though the firewall than say, SMTP connections).

It turns out that VMware Studio apparently cannot create virtual appliances with virtual floppy drives, even if you use the command-line tools (if that's wrong, please e-mail me--the documentation doesn't seem to indicate how to do it).

My next idea was to create an additional, very small, hard disk drive and write the output to that. This actually works in practice, but it's very cumbersome to retrieve data from. We need to import the returned .vmdk to one of our VMs, which then needs to be power-cycled so it can mount the disk and retrieve the data. I went looking for easier solutions for mounting .vmdk files and found references to a VMware Disk Mount Utility, but unfortunately the most recent version was shipped with Workstation 5.5 and appears to not read virtual hardware rev 4 .vmdk files created with ESX(i).

I then found signs pointing to the VMDKmounter utility on Mac OS X, which excited me quite a lot since I use a Mac and this would make the data retrieval trivially easy. Unfortunately this utility relies on MacFUSE, which has not yet been updated to handle 64-bit kernels. I'm running OS 10.6.2 with a 64-bit kernel. Damn.

This basically means my best option for grabbing a plain text file off a .vmdk is to import it to a VM and reboot. WTF? There has to be an easier way to do this.

Second post

2010-01-18 16:25:50 by chort

Testing new Blogsum-based blog to rant about stuff that annoys me and possibly suggest solutions.