Recently we came across an issue while integrating EBS R12 with OAM 11g using 10g Webgate for one of our client. For detail step by step integration process please check our book  at  Oracle E-Business R12 integration with OID/OAM

On requesting the EBS URL , User was redirected to Authenticate Agent URL value  which is set as Profile option in EBS . This request was then intercepted by 10g Webgate on OHS and forwarded to OAM.
OAM after checking that  the requested resource is protected was suppose to apply the protected authentication policy and present the challenge page to User.
But instead of getting the Challenge page, user was experiencing HTTP 404 error:-

On enabling HTTP headers trace, it shows that on requesting /ssologin , OAM server responds back with 404 error:-

GET /ebsauth_visr12/ssologin HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/x-ms-application, application/x-ms-xbap, application/, application/xaml+xml, application/, application/, application/msword, */*

HTTP/1.1 302 Moved Temporarily

GET /oam/server/obrareq.cgi?wh%3Dvisr12_hostid %20wu%3D%2F ebsauth_visr12%2F ssologin %20wo%3D1%20rh%3D http%3A%2F%2F innowavexx 2%252F ssologin HTTP/1.1
If-None-Match: “9696-8-4c626e271ac40″
Connection: Keep-Alive

HTTP/1.1 404 Not Found
Server: Oracle-Application-Server-11g
Content-Length: 188
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

Under Oam diagnostic log file ( default location- $DOMAIN_HOME/servers/oam_server1/logs) , error was recorded as:-

oam_server1-diagnostic.log:[2012-08-01T11:16:24.338+01:00] [oam_server1] [TRACE] [] [oracle.oam.engine.policy] [tid: NioProcessor-1] [userId: <anonymous>] [ecid: 0000JZT1WPXBDCHMyufd6G1G5x0T000003,0] [APP: oam_server] [SRC_METHOD: fine] [SRC_CLASS:] Called isResourceProtected () with resource : Resource Name: WebGateResource Type: HTTP URL: /ssologin RESULT DENY

Which clearly states that its trying to access /ssologin which is being denied by OAM and hence 404 error.

Reason – The issue was happening because of ‘DENY on Not Protected‘ flag on webgate agent registered with OAM.
The difference between 10g and 11g OAM is that in 11g, the default is that everything is protected once the webgate is installed (deny on protected is checked).
In 11g if you want a specific url to be not protected, you may need to create a public resource policy and protected via an anonymous auth scheme which will stop it from being protected.
In 10g the default behavior is to set the resource as allowed unless a specific policy is created for that resource.

If the ‘Deny On Not Protected’ flag is set in the Agent settings, HTTP-404 Not Found error occurs in the browser when the resource is accessed.

Solution – Uncheck the ‘DENY On Not Protected’ flag under oam console:-
1) Login to OAM Console http://oam_hostname:adminserver_port/oamconsole
2) System Configuration –>Access Manager Agents –> SSO Agents –> OAM Agents –> search for respective Agent
3)Uncheck ‘DENY on NOT Protected’
4)click Apply and Bounce the OHS of respective webgate