{"id":5301,"date":"2013-03-27T07:18:14","date_gmt":"2013-03-27T11:18:14","guid":{"rendered":"http:\/\/onlineappsdba.com\/?p=5301"},"modified":"2013-03-27T08:05:06","modified_gmt":"2013-03-27T12:05:06","slug":"oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable","status":"publish","type":"post","link":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/","title":{"rendered":"OAM server gets shut down automatically : fork: Resource temporarily unavailable"},"content":{"rendered":"

Hi All,<\/p>\n

Today I \u2018m very happy to share some information regarding a show stopper issue which I faced few days back in production box.<\/p>\n

The issue was\u00a0 when the load gets increased in the server , OAM server gets shut down Automatically.<\/p>\n

For more information, Please go through below :<\/p>\n

Problem Description:<\/strong><\/p>\n

We are using 2 single sign on solutions in which OAM is the centralized authentication server for tracks like portal,Siebel,OBIZ etc. .<\/p>\n

    \n
  1. OAS SSO 10.1.4.3<\/li>\n
  2. OAM 10.1.4.3<\/li>\n<\/ol>\n

    The problem was when the load gets increased in the server , OAM server gets shut down automatically.<\/p>\n

    Error details:<\/strong><\/p>\n

    –<\/span>bash-3.00$ \/u01apps\/oaminstall\/access\/oblix\/apps\/common\/bin\/access_server_watchdog: fork: Resource temporarily unavailable<\/span><\/p>\n

    \/u01apps\/oaminstall\/identity\/oblix\/apps\/common\/bin\/ois_server_watchdog: fork: Resource temporarily unavailable<\/span><\/p>\n

    Access Server has stopped running. Starting a new instance.<\/span><\/p>\n

    OIS Server has stopped running. Starting a new instance.<\/span><\/p>\n

    \/u01apps\/oaminstall\/identity\/oblix\/apps\/common\/bin\/start_ois_server<\/span><\/p>\n

    \/u01apps\/oaminstall\/access\/oblix\/apps\/common\/bin\/start_access_server: fork: Resource temporarily unavailable<\/span><\/p>\n

    \/u01apps\/oaminstall\/identity\/oblix\/apps\/common\/bin\/ois_server_watchdog: fork: Resource temporarily unavailable<\/span><\/p>\n

    Could not start the Access Server. The watchdog is stopping.<\/span><\/p>\n

    \/u01apps\/oaminstall\/identity\/oblix\/apps\/common\/bin\/start_ois_server: fork: Resource temporarily unavailable<\/span><\/p>\n

    \/u01apps\/oaminstall\/identity\/oblix\/apps\/common\/bin\/start_ois_server: fork: Resource temporarily unavailable<\/span><\/p>\n

    Error: Could not determine your OS. Aborting.<\/span><\/p>\n

    Access Server has stopped running. Starting a new instance.<\/span><\/p>\n

    Using Linux Threading Library.<\/span><\/p>\n

    For Past 2 weeks , I was breaking my head to fix up showstopper issue. Lots of workaround & alternate solution approach was buzzing up in my mind \u00a0and I was cracking in all possible ways whether the Issue might have happened \u00a0because of\u00a0 Hardware sizing or memory space issue \/Loading or Version Problem or Connection Reached to maximum Limits but I never thought that a single change in the server will end the showstopper issue easily J<\/p>\n

    \u00a0I would like to share\u00a0 the point\u00a0 which I covered during analyze.<\/p>\n

      \n
    1. Verified soft and hard limits for file descriptors and number of processes available to a users.<\/li>\n<\/ol>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Result: Limit value looks pretty okay and less possibility for occurrence of issue<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a02.\u00a0\u00a0 As a rule of thumb, \u00a0verified on the firewall timeout the Maximum Client Session Time\u00a0should be slightly less than firewall inactivity connection timeout value.<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Result: Changed Accordingly , but still the problem persists<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0 3.\u00a0\u00a0As a rule of thumb, verified on the OAM component version, Access Server and Identity Server need to have version greater than or equal than webgate version.<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Result: Applied with latest version BP13 but same problem persists.<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a04.\u00a0 Verified the resource policies for all tracks.<\/p>\n

      \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Result: Configuration looks pretty okay and less possibility for occurrence of issue<\/p>\n

      I was wondering & didn\u2019t get any clue where the issue persist . Few days before when I was reading oracle trouble shooting document (http:\/\/docs.oracle.com\/cd\/E15217_01\/doc.1014\/e12489\/trblsht.htm<\/a> .)<\/p>\n

      Immediately something strike in my mind , I got some clue and suspected that the issue might have happened \u00a0due to F.10 NPTL Requirements and Post-Installation Tasks<\/p>\n

      Here by the solution :<\/p>\n

      Solution<\/strong>: The native libraries support something called NPTL which stands for “Native POSIX Thread Library. This library allows Linux to run multithreaded program much faster regardless. the program is 32 bit or 64 bit. The Linux 5 OS comes with this library by default.<\/p>\n

      Start identity Server using start_ois_server_nptl instead of start_ois_server<\/p>\n

      Start Access Server using start_access_server_nptl instead of start_access_server<\/p>\n

      Note<\/strong>:\u00a0We were used to start the server without NPTL.<\/p>\n

      These NPTL scripts use the new multithreaded NPTL capable libraries, resulting better performance.<\/p>\n

      Hope this post would help you if you are facing same kind of issue. \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"

      Hi All, Today I \u2018m very happy to share some information regarding a show stopper issue which I faced few […]<\/p>\n","protected":false},"author":115,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[212,40,42,70],"tags":[],"class_list":["post-5301","post","type-post","status-publish","format-standard","hentry","category-performance","category-sso","category-troubleshooting","category-webgate"],"yoast_head":"\nOAM server gets shut down automatically : fork: Resource temporarily unavailable -<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OAM server gets shut down automatically : fork: Resource temporarily unavailable -\" \/>\n<meta property=\"og:description\" content=\"Hi All, Today I \u2018m very happy to share some information regarding a show stopper issue which I faced few […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/\" \/>\n<meta property=\"article:published_time\" content=\"2013-03-27T11:18:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2013-03-27T12:05:06+00:00\" \/>\n<meta name=\"author\" content=\"Masroof Ahmad\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Masroof Ahmad\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/\",\"url\":\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/\",\"name\":\"OAM server gets shut down automatically : fork: Resource temporarily unavailable -\",\"isPartOf\":{\"@id\":\"https:\/\/onlineappsdba.com\/#website\"},\"datePublished\":\"2013-03-27T11:18:14+00:00\",\"dateModified\":\"2013-03-27T12:05:06+00:00\",\"author\":{\"@id\":\"https:\/\/onlineappsdba.com\/#\/schema\/person\/909a876ed58d400faf82caf81d61bfdb\"},\"breadcrumb\":{\"@id\":\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/onlineappsdba.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OAM server gets shut down automatically : fork: Resource temporarily unavailable\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/onlineappsdba.com\/#website\",\"url\":\"https:\/\/onlineappsdba.com\/\",\"name\":\"\",\"description\":\"Oracle Implementation & Training Experts\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/onlineappsdba.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/onlineappsdba.com\/#\/schema\/person\/909a876ed58d400faf82caf81d61bfdb\",\"name\":\"Masroof Ahmad\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/onlineappsdba.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/10f9db7bdbbd7f9ccfbe9b2d208e5978fc28315e9c704383e639a926ea0fce5f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/10f9db7bdbbd7f9ccfbe9b2d208e5978fc28315e9c704383e639a926ea0fce5f?s=96&d=mm&r=g\",\"caption\":\"Masroof Ahmad\"},\"url\":\"https:\/\/onlineappsdba.com\/index.php\/author\/masroof\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"OAM server gets shut down automatically : fork: Resource temporarily unavailable -","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/","og_locale":"en_US","og_type":"article","og_title":"OAM server gets shut down automatically : fork: Resource temporarily unavailable -","og_description":"Hi All, Today I \u2018m very happy to share some information regarding a show stopper issue which I faced few […]","og_url":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/","article_published_time":"2013-03-27T11:18:14+00:00","article_modified_time":"2013-03-27T12:05:06+00:00","author":"Masroof Ahmad","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Masroof Ahmad","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/","url":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/","name":"OAM server gets shut down automatically : fork: Resource temporarily unavailable -","isPartOf":{"@id":"https:\/\/onlineappsdba.com\/#website"},"datePublished":"2013-03-27T11:18:14+00:00","dateModified":"2013-03-27T12:05:06+00:00","author":{"@id":"https:\/\/onlineappsdba.com\/#\/schema\/person\/909a876ed58d400faf82caf81d61bfdb"},"breadcrumb":{"@id":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/onlineappsdba.com\/index.php\/2013\/03\/27\/oam-server-gets-shut-down-automatically-fork-resource-temporarily-unavailable\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/onlineappsdba.com\/"},{"@type":"ListItem","position":2,"name":"OAM server gets shut down automatically : fork: Resource temporarily unavailable"}]},{"@type":"WebSite","@id":"https:\/\/onlineappsdba.com\/#website","url":"https:\/\/onlineappsdba.com\/","name":"","description":"Oracle Implementation & Training Experts","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/onlineappsdba.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/onlineappsdba.com\/#\/schema\/person\/909a876ed58d400faf82caf81d61bfdb","name":"Masroof Ahmad","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/onlineappsdba.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/10f9db7bdbbd7f9ccfbe9b2d208e5978fc28315e9c704383e639a926ea0fce5f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/10f9db7bdbbd7f9ccfbe9b2d208e5978fc28315e9c704383e639a926ea0fce5f?s=96&d=mm&r=g","caption":"Masroof Ahmad"},"url":"https:\/\/onlineappsdba.com\/index.php\/author\/masroof\/"}]}},"_links":{"self":[{"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/posts\/5301","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/users\/115"}],"replies":[{"embeddable":true,"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/comments?post=5301"}],"version-history":[{"count":0,"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/posts\/5301\/revisions"}],"wp:attachment":[{"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/media?parent=5301"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/categories?post=5301"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/onlineappsdba.com\/index.php\/wp-json\/wp\/v2\/tags?post=5301"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}