[WEB SECURITY] Need a vulnerable XML Web Service

Ory Segal SEGALORY at il.ibm.com
Wed Jun 9 04:27:13 EDT 2010


Arian,

There seem to be a gap between what you are seeing, and what I am 
hearing/seeing. Maybe it is a difference in the type of customers? We have 
quite a few customers, who are constantly using our SOAP tool w/ AppScan, 
to scan SOA environments, which include SOAP and WS-security protocols.

In addition, I often see Applets, Flex/Flash and .NET code that talks 
"binary protocols" over HTTP (Java object serialization, AMF and 
Microsoft's Binary format for SOAP respectively). 

SQL Injection and all sorts of information leakage, are the most common 
issues I have found, although I have seen a few DoS scenarios, in some of 
these parsers/serializers.

Not sure if you call these "web services", but in recent years, I am 
seeing more and more AJAX-based web applications, which expose JSON-based 
services. IBM uses Dojo for many of its new products, and that's common 
practice.

-Ory

-------------------------------------------------------------
Ory Segal
Security Products Architect
AppScan Product Manager
Rational, Application Security
IBM Corporation
Tel: +972-9-962-9836
Mobile: +972-54-773-9359
e-mail: segalory at il.ibm.com 




From:   "Arian J. Evans" <arian.evans at anachronic.com>
To:     Ory Segal/Haifa/IBM at IBMIL
Cc:     7Lyrix <7lyrix at gmail.com>, Arshan Dabirsiaghi 
<arshan.dabirsiaghi at aspectsecurity.com>, Nilesh Bhosale 
<nilesh at gslab.com>, Tom Stripling <tstripling at appsecconsulting.com>, 
websecurity at webappsec.org
Date:   09-06-2010 03:28 AM
Subject:        Re: [WEB SECURITY] Need a vulnerable XML Web Service
Sent by:        arian.evans at gmail.com



Ory,

My point was that - from the pool of web services I have seen - 
standards-based SOAP Web Services and "Real World" rarely go together. I 
haven't seen many SOAP WS at all. I've seen a large pool of SOAP-like WS, 
and a larger pool of non-SOAP WS.

Blind and n-Order (2nd, 3rd, etc.) SQL Injection are a web services 
problem - I hope I was clear about that. I just do not believe you can 
measure exploitability effectively with a BB scanner.

AuthC/Z are also significant web services security problems. I just do not 
believe you can measure them effectively with a BB scanner.

I have not seen many cases where WS-Security SOAP standards compliance, or 
lack of, is the core security problem with a web service.

There are a large number of technology designs and implementations that 
are called "web services" today. The two things they have in common are:

1. HTTP: They use the HTTP protocol to transfer data, and likely commands.

2. Not a Web Page: They don't run natively in a web browser. (they need a 
plugin, like Flash, or something proprietary)

Quite a few I've seen use proprietary binary data streams, message 
formats, encryption, encoding, etc. to transfer data from one WS to a 
consumer (sometimes another web service, sometimes a legacy 
machine/service/MQ, and sometimes a proprietary client implementation).

The world of web services is very bespoke, and just ain't like the world 
of webapps. WS vary a lot.

Now - I am all for building standards-based test cases. Nothing wrong with 
that.

What I would really like to see is some statistics and numbers from the 
parties on this list about:

1. How many web services have you seen?
2. What technologies do they use? (protocol, standards, format)
3. What type of vulnerabilities do you actually discover on them?
4. What issues can be found in a default scan vs. which issues require 
expert configuration of the scanner?

That would be interesting data, and help folks prioritize what to put into 
test apps like WebGoat, and provides data to correct any biases that the 
same of WS I've seen include.

---
Arian Evans


On Sat, Jun 5, 2010 at 11:44 PM, Ory Segal <SEGALORY at il.ibm.com> wrote:
Hi, 

This brings us back to my original-original message - there is a need to 
have a real-world sample vulnerable SOAP web service, hopefully installed 
with one of the vulnerable web applications (WebGoat, etc.), so that 
people can actually learn about the *real* problems. Current sample 
vulnerable web services, tend to concentrate on the easy things like SQLi 
over SOAP, which as you have clearly stated, is almost never the actual 
vulnerabilities you see in the wild, right? 

And that was my original point :-) 

-Ory 

-------------------------------------------------------------
Ory Segal
Security Products Architect 
AppScan Product Manager
Rational, Application Security
IBM Corporation
Tel: +972-9-962-9836
Mobile: +972-54-773-9359 




From:        "Arian J. Evans" <arian.evans at anachronic.com> 
To:        Ory Segal/Haifa/IBM at IBMIL 
Cc:        7Lyrix <7lyrix at gmail.com>, Arshan Dabirsiaghi <
arshan.dabirsiaghi at aspectsecurity.com>, Nilesh Bhosale <nilesh at gslab.com>, 
Tom Stripling <tstripling at appsecconsulting.com>, websecurity at webappsec.org 

Date:        05-06-2010 11:16 PM 
Subject:        Re: [WEB SECURITY] Need a vulnerable XML Web Service 
Sent by:        arian.evans at gmail.com 



Ory,

Actually, the 85% was somewhat anecdotal. The 1% to 15% was not anecdotal, 
and based upon vuln count, pulse and frequency, discovered per input, 
based upon expected issues compared to per-input web application discover, 
and issues uncovered from human review. We are extremely quantitative and 
stats driven at our shop - you should know that. :)

Given the state of WS scanning, simply the act of parsing and manipulating 
complex SOAP messages is an accomplishment in and of itself. Like I said - 
you may be "state of the art" here, but I don't think that deserves 
bragging rights in terms of the results folks get.

I don't tend to agree that it beats manual fuzzing. My experience here is 
that scanners have limited value at best.

I have not seen your scanner in a few years, but I have seen the results, 
and I have seen a pretty sizable sample of web services from which I base 
my opinion.

You have probably put a lot of hard work over the years to get your 
scanner to the point it is at today, and it may beat all the other desktop 
scanners hands-down. Which naturally is going to make you proud of it, and 
proud of the work you've done, and there's nothing wrong with that.

But in terms of my thoughts on functionality utility and results - I 
stated my case in my previous message.

If you want to improve the discussion and correct any errors in my 
statements - start publishing analysis details and data. What vulns do you 
find, how many, where? What type of web services do you see, how many can 
you actually work on, who really uses SOAP, etc. etc.?

We have shied away from default BB scanning of web services for these 
reasons:

1) A significant number were bespoke -e.g.-followed no standard known to 
man, or loosely followed some standard(s) at best. Only a subset use SOAP.

2) Little to now value from default (lame) scanning

3) Fairly significant cost to customize messaging and communication for 
each bespoke web service to effectively "scan" for still relatively 
minimal results gain

4) Inability to find some of the biggest, glaring holes (AuthC/Z) which 
are in comparison relatively easy to find in a web app via automation 
(even though I don't think the desktop webapps scanners do this well at 
all, today)

Well, my list could go on, but I think it is all more of the same initial 
point I made.

If I'm not thinking about this correctly, missing something obvious, 
please don't hesitate to correct me. Large samples of data help too. :)

One thing I have to compliment IBM on - they have at least been honest 
when the marketing runs out of control in webappsec. What was it, a year 
or two ago, that the Proventia device claimed to "block" a bunch of the 
WASC TC/24 items, and even prevent "phishing"?

When we questioned these "features" their product management folks came 
forward and apologized for marketing zeal, and reality-grounded the 
message. Very impressive,

---
Arian Evans
Software Security Scanner Sophisticate


On Sat, Jun 5, 2010 at 11:38 AM, Ory Segal <SEGALORY at il.ibm.com> wrote: 
Arian, 

Are you sure it's 85% and not 87.5%? you sure know how to make things 
sound scientific ;-) 

Hey - we (IBM) invest a lot of effort in AppScan's SOAP testing 
capabilities. Yes it is limited, but it beats the hell out of doing a lot 
of manual SOAP fuzzing, and it does take care of most WS-Security 
requirements, which removes the burden of doing things yourself. 

Naturally, we can't validate the results of the testing on the back-end 
systems, but that's the nature of blackbox scanning. 

As I've said - unlike other scanners, that simply read WSDLs and send some 
lame SOAP messages, we do invest a lot of effort in being able to create, 
analyze and submit more complex SOAP messages (WS-Addressing, WS-Security, 
certificates, etc.), you should really give it a try sometime. 

-Ory 


-------------------------------------------------------------
Ory Segal
Security Products Architect 
AppScan Product Manager
Rational, Application Security
IBM Corporation
Tel: +972-9-962-9836
Mobile: +972-54-773-9359
e-mail: segalory at il.ibm.com 




From:        "Arian J. Evans" <arian.evans at anachronic.com> 
To:        Arshan Dabirsiaghi <arshan.dabirsiaghi at aspectsecurity.com> 
Cc:        Ory Segal/Haifa/IBM at IBMIL, Tom Stripling <
tstripling at appsecconsulting.com>, 7Lyrix <7lyrix at gmail.com>, Nilesh 
Bhosale <nilesh at gslab.com>, websecurity at webappsec.org 
Date:        05-06-2010 01:18 AM 
Subject:        Re: [WEB SECURITY] Need a vulnerable XML Web Service 
Sent by:        arian.evans at gmail.com 




For once I have absolutely agree with Arshanti from Aspect. Except for his 
unfounded praise of Black Box scanners on web services.

Let's be blunt here gentlemen: scanners are between 85% ineffective to 
utterly useless for testing web services.

Scanner Success Scenario: The only scenario where scanners work reasonably 
well is when a RESTful web service is bound to an identical UI, as some 
modern frameworks automatically provide (say, Drupal or Ruby on Rails). 
This way you can trace data, output, and behavioral changes via monitoring 
the UI. However - this does not ensure that the controls (like AuthC/Z) 
test are the same between the API and the UI.

Web Service Test Requirements: In the vast majority of test-cases for web 
services -- there are important unidirectional functions whose security 
implications are not visible or measurable through a response from the web 
service (which BB scanners ultimately rely on).

So, sure, your scanner might "hack the crap out of it" but you and your 
scanner will never know, unless you go look in the async MQs, the DBs, the 
logs, etc. etc. and see what is there, and what transactions actually 
occurred. Scanners by and large cannot provide this insight, and in my 
experience most scanner jockeys rarely go and perform deeper analysis, nor 
know how to.

Web Service Scanning Results: With BB scanning you can find a small subset 
of verbose SQLi and directory traversal, and that's about it. There are 
edge cases where you can find more, but by and large you will find little 
to nothing. That's a fact.

Web Service "Standards": Not to mention that many, many web services do 
not follow standards, so you have to customize your BB scanner for each 
one, which most BB scanners aren't very good at handling.

Web Service Authentication Holes: The biggest issues with web services are 
AuthC/Z issues. Syntax injection issues are there too, but nothing spells 
Security-Fail like AuthC issues which are fairly common in WS from what 
I've seen. But don't take my word for it - talk to the folks on WASC and 
OWASP who specialize in testing web services and they will tell you the 
same thing.

Source Code Scanners on Web Services: Source code scanners are fairly 
limited here for different reasons, but the outcome is usually the same. 
Source code scanners are better at finding potential SQLi than BB/runtime 
scanners, but they also tend to fail at finding AuthC/Z issues and design 
pattern problems out of the box. (some source code scanners can be 
customized to do this).

Binary Analysis on Web Services: I would expect binary analysis to be 
similarly, but more-so, limited than source code scanners, due to 
limitations in tracing regression and such. Basically the way most web 
services are used are as a transaction and message broker that is part of 
a larger, distributed, and complex system, and analyzing these for 
security implications effectively requires the sum of the parts, which is 
not something binary analysis is very good at/for.


Conclusion: So what are we left with?

I recommend people perform the following on web services:

+ Threat modeling, especially on B2B or WS to WS, to make sure you 
understand your attack surface

+ Human source-code review, to review your controls implementation 
particularly AuthC/Z

+ Run-time analysis to see how the running engine handles input thrown 
into it

+ A customized source-code scanner might be useful here for repeated 
testing over time


Deep Thoughts from a Black Box Scanner Builder: At my work we have to say 
that we "scan web services" for business reasons. Basically to check off 
the check-box on RFPs that we respond to. We have to do this thanks to all 
the Scanner-Spin-Doctors out there inflating claims about what value they 
provide when scanning web services with their Windows-based desktop 
scanning widgets. (Or the hosted "SaaS" versions of the the same widgets.)

However - while we say "yes we scan web services" - We make sure we are 
honest to our customers about what value we provide.

Web Service Scanning Value: We tell people that they will get between 1% 
and 15% of the assessment depth & coverage (value) using our scanning 
engine on a web service, that they would expect to get from using it on a 
website. And we even customize the web service scanning for their unique 
WS messaging types, and know what we are doing, and even then that is 
about the best you can do with a scanner.

Most people appreciate our honestly, and that we discourage our customers 
from using our scanning engine on web services unless they really 
want/need to.

The quality of our black-box web scanning engine is excellent, and yes -- 
it supports "SOAP messages", and more importantly -- also supports and the 
even more-common "SOAP-LIKE Messages" :). It also provides coverage 
features that no desktop widget provides today - and we are very proud of 
the quality of coverage it provides on Websites. Because of this we feel 
that it is a bit of  a sham to provide exceptionally high-quality coverage 
scanning web applications and then turn around and provide unequally 
low-quality coverage for web services under the same brand name.

Most people appreciate honesty from what we see. We've only had a few 
folks wander away to marketing misdirection from magic web service 
scanning products.

Since all scanners suck at testing web services, it's easy to spin this 
and for a given scanner make to say they have "industry leading security 
scanning coverage for web services" and actually be honest about it. 
Because they all suck pretty equally.

But it doesn't change that fact it is utterly lame and misleading, and a 
bit manipulative of consumer expectations.

Happy Friday!

---
Arian Evans
Software Security Scanner Sycophant



On Fri, Jun 4, 2010 at 11:31 AM, Arshan Dabirsiaghi <
arshan.dabirsiaghi at aspectsecurity.com> wrote: 
AppScan does a great job, I?m sure. Recently, though, I?ve had success 
using SoapUI (free version) for testing complex, WS-Security-protected 
endpoints. It?s got Groovy scripting capability that I?ve used to turn a 
few man days of scripting into good, automated, repeatable 
authentication/authorization tests for a whole suite of Web Services. 
Assertions are a big part of their framework and they make the tests easy 
to write. Naturally, injection testing is a little different. 
  
Sure, you could hook it up to a fuzz database pretty easily to launch 
injection attacks along the same line. Unfortunately, verifying the 
results of those tests is something at which AppScan will (probably) be 
much better. Can?t say Web Service injection testing has every been very 
fruitful for me, though. Karaoke-quality assessors will look for simple 
input reflected in output, but that is meaningless 99% of the time, since 
the response data is usually serialized reliably and not hand built, and 
couldn?t be exploited even if it was. SQL injection is pretty much the 
only hope you have for meaningful results at the time of the test. 
  
Assessments are usually scoped in such a way that the app that consumes 
your Web Service input is not being tested at the same time as the Web 
Service itself. So, your opportunity to observe evidence of a successful 
attack is slim. Said attacks are probably way downstream but probably have 
a better than average chance at success, if I had to guess. 
  
Arshan 
  
From: Ory Segal [mailto:SEGALORY at il.ibm.com] 
Sent: Friday, June 04, 2010 6:11 AM
To: Tom Stripling
Cc: '7Lyrix'; 'Nilesh Bhosale'; websecurity at webappsec.org 
Subject: RE: [WEB SECURITY] Need a vulnerable XML Web Service 
  
AFAIK, You have two options, one is a product that my company sells - 
[ALERT!!! - propaganda, if you don't want to read, skip to part (B)] 

(A) Luckily for me, [sorry about the shameless propaganda] I am using IBM 
Rational AppScan, which comes with a SOAP tool called GSC (Generic SOAP 
Client). GSC supports WS-Security 1.1, SOAP attachments, WS-Addressing, 
certificates, etc. 

In addition, AppScan itself is capable of testing SOAP messages.  
http://www-01.ibm.com/software/awdtools/appscan/ 


(B) As far as I know, the only other tool that is capable of using 
WS-Security and other WS-* is SOAPUI ( http://www.soapui.org/ ), but it 
doesn't have the same security testing capabilities as AppScan 

-Ory 

-------------------------------------------------------------
Ory Segal
Security Products Architect 
AppScan Product Manager
Rational, Application Security
IBM Corporation
Tel: +972-9-962-9836
Mobile: +972-54-773-9359
e-mail: segalory at il.ibm.com 




From:        "Tom Stripling" <tstripling at appsecconsulting.com> 
To:        Ory Segal/Haifa/IBM at IBMIL, "'7Lyrix'" <7lyrix at gmail.com>, <
websecurity at webappsec.org> 
Cc:        "'Nilesh Bhosale'" <nilesh at gslab.com> 
Date:        04-06-2010 01:02 AM 
Subject:        RE: [WEB SECURITY] Need a vulnerable XML Web Service 




Along that vein, most penetration testing tools I?ve used are awful at 
constructing WS-Security headers for manual testing.  Excluding 
high-dollar automated scanners (I?m not trying to start a vendor war 
here), what tools do you all use for testing web services when 
WS-Security, SAML, etc. are being used?  I have yet to find a free or 
cheap tool that allows me to test these effectively.  I sometimes have to 
resort to building a testing client myself just so I can interact with the 
web service manually. 
  
  
From: Ory Segal [mailto:SEGALORY at il.ibm.com] 
Sent: Thursday, June 03, 2010 1:39 PM
To: 7Lyrix; websecurity at webappsec.org
Cc: Nilesh Bhosale
Subject: Re: [WEB SECURITY] Need a vulnerable XML Web Service 
  
Has anyone noticed how all these "theoretical / tutorial" Web Services, 
never use any WS-Security? 

a) Real world SOAP web services, in real SOA environments, usually come 
with plenty of WS-Security 

b) There are plenty of things that can go wrong when implementing 
WS-Security, yet most security experts and most demo web sites, tend to 
talk about SQL-Injection over SOAP. I've seen some External entities here 
and there, but that's as deep as it goes most times. 

-Ory 

-------------------------------------------------------------
Ory Segal
Security Products Architect 
AppScan Product Manager
Rational, Application Security
IBM Corporation
Tel: +972-9-962-9836
Mobile: +972-54-773-9359 




From:        7Lyrix <7lyrix at gmail.com> 
To:        Nilesh Bhosale <nilesh at gslab.com> 
Cc:        websecurity at webappsec.org 
Date:        03-06-2010 09:17 PM 
Subject:        Re: [WEB SECURITY] Need a vulnerable XML Web Service 
  





A few.
See:
http://www.yehg.net/lab/pr0js/training/webgoat.php#Web_Services


On Thu, Jun 3, 2010 at 3:17 PM, Nilesh Bhosale <nilesh at gslab.com> wrote:
> Thanks for all the responses.
>
> Does Webgoat has a vulnerable XML Webservice?
>
> Thanks,
> Nilesh
>
> On Thursday 03 June 2010 11:36 AM, 7Lyrix wrote:
>
> Try webgoat:
> http://www.owasp.org/index.php/Category:OWASP_WebGoat_Project
>
> Ivan Buetler, thank you for http://www.hacking-lab.com. It's very great.
>
> On Wed, Jun 2, 2010 at 3:23 PM, Nilesh Bhosale <nilesh at gslab.com> wrote:
>
>
> I need a vulnerable Web Service which I can use to try out and learn 
some of
> the XML web service attacks.
>
> Thanks,
> Nilesh
>
> 
----------------------------------------------------------------------------
> Join us on IRC: irc.freenode.net #webappsec
>
> Have a question? Search The Web Security Mailing List Archives:
> http://www.webappsec.org/lists/websecurity/archive/
>
> Subscribe via RSS: http://www.webappsec.org/rss/websecurity.rss [RSS 
Feed]
>
> Join WASC on LinkedIn
> http://www.linkedin.com/e/gis/83336/4B20E4374DBA
>
>
>
>
>
>

----------------------------------------------------------------------------
Join us on IRC: irc.freenode.net #webappsec

Have a question? Search The Web Security Mailing List Archives: 
http://www.webappsec.org/lists/websecurity/archive/

Subscribe via RSS: 
http://www.webappsec.org/rss/websecurity.rss [RSS Feed]

To unsubscribe email websecurity-unsubscribe at webappsec.org and reply to 
the confirmation email

Join WASC on LinkedIn
http://www.linkedin.com/e/gis/83336/4B20E4374DBA 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20100609/319e22f2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2359 bytes
Desc: not available
URL: <http://lists.webappsec.org/pipermail/websecurity_lists.webappsec.org/attachments/20100609/319e22f2/attachment.gif>


More information about the websecurity mailing list