[WEB SECURITY] program to crawl website looking for string patterns

MaXe owasp at intern0t.net
Mon Sep 19 13:22:20 EDT 2011


And if the website is public, some Google Kung Fu:
site:domain-name.tld keyword

That is of course, only what Google has seen and it will also only work if Google is not disabled from viewing the site.



~ MaXe
----- Original meddelelse -----
> This may also work:
> https://github.com/urbanadventurer/WhatWeb/
> 
> On Fri, Sep 16, 2011 at 2:25 PM, Tasos Laskos <tasos.laskos at gmail.com>
> wrote:
> > <shameless selfpromotion>
> > 
> > Or you could use arachni[1]:
> > 
> > 1) Create a module like:
> > ----------------
> > module Arachni
> > module Modules
> > class MyModule < Arachni::Module::Base
> > 
> >    def initialize( page )
> >        @page = page
> >    end
> > 
> >    def run( )
> >        match_and_log( "the string you're looking for" )
> >    end
> > 
> >    def self.info
> >        {
> >            :name           => 'My module',
> >            :description    => %q{Greps pages for a string.},
> >            :author         => 'Your name',
> >            :version        => '0.1',
> >            :targets        => { 'Generic' => 'all' },
> >            :issue   => {
> >                :name        => %q{Found my string},
> >                :description => %q{some description},
> >                :cwe         => '',
> >                :severity    => Issue::Severity::LOW,
> >                :cvssv2      => '0',
> >                :remedy_guidance    => %q{Remode the damn thing.},
> >                :remedy_code => '',
> >            }
> >        }
> >    end
> > 
> > end
> > end
> > end
> > ----------------
> > 2) Save it as "my_module.rb" and put it under "modules/recon/grep/"
> > 3) run arachni like so:
> >        arachni -m my_module <site url>
> > 
> > 
> > And you're good to go. :)
> > 
> > [1] http://arachni.segfault.gr/
> > 
> > </shameless selfpromotion>
> > 
> > On 09/16/2011 07:51 PM, Ryan Dewhurst wrote:
> > > 
> > > w3af [0] has lot's of grepping plugins which can easily be expanded.
> > > Should do what you want.
> > > 
> > > [0] http://w3af.sourceforge.net/
> > > 
> > > Ryan Dewhurst
> > > 
> > > blog www.ethicalhack3r.co.uk
> > > projects www.dvwa.co.uk | www.webwordcount.com
> > > twitter www.twitter.com/ethicalhack3r
> > > 
> > > 
> > > 
> > > On Fri, Sep 16, 2011 at 2:55 PM, Youngquist, Jason R.
> > > <jryoungquist at ccis.edu>  wrote:
> > > > 
> > > > We are looking for a tool that can be configured to crawl for
> > > > string patterns (ie. SSNs, credit card numbers, etc).  Cornell's
> > > > Spider 2008 beta has this capability, but every time we used it,
> > > > it crashed on us.
> > > > 
> > > > We also found a program called webshag, but it would only look for
> > > > pre-defined stuff like email addresses or external links.
> > > > 
> > > > Did some googling, but haven't really found anything.  Thoughts?
> > > > 
> > > > 
> > > > 
> > > > Thanks.
> > > > Jason Youngquist, CISSP
> > > > Information Technology Security Engineer
> > > > Technology Services
> > > > Columbia College
> > > > 1001 Rogers Street, Columbia, MO  65216
> > > > (573) 875-7334
> > > > jryoungquist at ccis.edu
> > > > http://www.ccis.edu
> > > > 
> > > > 
> > > > _______________________________________________
> > > > The Web Security Mailing List
> > > > 
> > > > WebSecurity RSS Feed
> > > > http://www.webappsec.org/rss/websecurity.rss
> > > > 
> > > > Join WASC on LinkedIn
> > > > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> > > > 
> > > > WASC on Twitter
> > > > http://twitter.com/wascupdates
> > > > 
> > > > websecurity at lists.webappsec.org
> > > > 
> > > > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
> > > > 
> > > 
> > > _______________________________________________
> > > The Web Security Mailing List
> > > 
> > > WebSecurity RSS Feed
> > > http://www.webappsec.org/rss/websecurity.rss
> > > 
> > > Join WASC on LinkedIn
> > > http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> > > 
> > > WASC on Twitter
> > > http://twitter.com/wascupdates
> > > 
> > > websecurity at lists.webappsec.org
> > > 
> > > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
> > > 
> > 
> > 
> > _______________________________________________
> > The Web Security Mailing List
> > 
> > WebSecurity RSS Feed
> > http://www.webappsec.org/rss/websecurity.rss
> > 
> > Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> > 
> > WASC on Twitter
> > http://twitter.com/wascupdates
> > 
> > websecurity at lists.webappsec.org
> > http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org
> > 
> 
> _______________________________________________
> The Web Security Mailing List
> 
> WebSecurity RSS Feed
> http://www.webappsec.org/rss/websecurity.rss
> 
> Join WASC on LinkedIn http://www.linkedin.com/e/gis/83336/4B20E4374DBA
> 
> WASC on Twitter
> http://twitter.com/wascupdates
> 
> websecurity at lists.webappsec.org
> http://lists.webappsec.org/mailman/listinfo/websecurity_lists.webappsec.org





More information about the websecurity mailing list