[NewGuide]Downloadthe2017GuidetoBigDataforTutorials,Solutions,andExclusiveSurveyResearch

FreeDownload

DealingwithStuckThreadson
WebLogic
bySpyrosDoulgeridis May.09,12IntegrationZone

LearnhowAPImanagementsupportsbetterintegrationinAchievingEnterpriseAgilitywith
MicroservicesandAPIManagement,broughttoyouinpartnershipwith3scale

WebLogicServerdiagnosesathreadasstuckifitiscontinuallyworking(notidle)foraset
periodoftime.
Youcantuneaserver'sthreaddetectionbehaviorbychangingthelengthoftimebeforeathreadis
diagnosedasstuck(StuckThreadMaxTime),andbychangingthefrequencywithwhichtheserver
checksforstuckthreads.CheckheretoseehowtochangetheStuckThreadMaxTime.

TheproblemorWhyareStuckThreadsevil?

WebLogicServerautomaticallydetectswhenathreadinanexecutequeuebecomes"stuck."Becausea
stuckthreadcannotcompleteitscurrentworkoracceptnewwork,theserverlogsamessageeachtimeit
diagnosesastuckthread.Ifallthreadsinanexecutequeuebecomestuck,theserverchangesitshealth
statetoeither"warning"or"critical"dependingontheexecutequeue:
Ifallthreadsinthedefaultqueuebecomestuck,theserverchangesitshealthstateto"critical."
(YoucansetuptheNodeManagerapplicationtoautomaticallyshutdownandrestartserversinthe
criticalhealthstate.Formoreinformation,see"NodeManagerCapabilities"inConfiguringand
ManagingWebLogicServer.)
Ifallthreadsinweblogic.admin.HTTP,weblogic.admin.RMI,orauserdefinedexecutequeue
becomestuck,theserverchangesitshealthstateto"warning."

Sopractically,acoupleofStuckThreadsmightnotcrashyourserverpreventingitfromservingrequest,
butitisabadsign.Usually,thenumberofstuckthreadswillincreaseandyourserver
willeventuallycrash.

Whatyoucandotoavoidyourapplicationcompletelyfail?
WebLogicServerchecksforstuckthreadsperiodically(thisistheStuckThreadTimerIntervaland
youcanadjustithere).Ifallapplicationthreadsarestuck,aserverinstancemarksitselffailed,if
configuredtodoso,exits.YoucanconfigureNodeManagerorathirdpartyhighavailabilitysolutionto
restarttheserverinstanceforautomaticfailurerecovery.Youcanconfiguretheseactionstooccurwhennot
allthreadsarestuck,butthenumberofstuckthreadshaveexceededaconfiguredthreshold:Shutdownthe
WorkManagerifithasstuckthreads.AWorkManagerthatisshutdownwillrefusenewworkandreject
existingworkinthequeuebysendingarejectionmessage.Inacluster,clusteredclientswillfailoverto
anotherclustermember.
Shutdowntheapplicationiftherearestuckthreadsintheapplication.Theapplicationisshutdown
bybringingitintoadminmode.AllWorkManagersbelongingtotheapplicationareshutdown,and
behaveasdescribedabove.
Marktheserverinstanceasfailedandshutitdownitdowniftherearestuckthreadsintheserver.In
acluster,clusteredclientsthatareconnectedorattemptingtoconnectwillfailovertoanother
clustermember.

Howtoidentifytheproblem?
Themostrecommendedwayistocheckthethreaddumps.CheckSendingEmailAlertForStuckThreads
WithThreadDumpspostofMiddlewaremagic,tohaveThreadDumpsmailedtoyouautomaticallywhen
theyoccur.
ToolstohelpyouwithanalyzingtheThreadDumpscanbe:
TDAThreadDumpAnalyzer
Samurai

Howtoworkaroundtheproblem?
AfteryouhaveidentifythecodethatcausestheStuckThread,thatisthecodewhichexecutiontakesmore
thantheStackThreadMaxTime,youcanuseWorkManagertoexecuteyourcode.WorkManagershavea
IgnoreStuckThreadoptionsthatgivestheabilitytoexecutelongrunningjobs.Seebelow:

BelowaresomepostsonhowtocreateaWorkManager

https://blogs.oracle.com/jamesbayer/entry/work_manage
r_leash_for_slow_js
http://jdeveloperfaq.blogspot.com/2011/05/faq34using
weblogicworkmanagersto.html

Test:HowtocreateaStuckThread?
HowtocreateaStuckThreadinordertotestyourweblogicsettings?Putabreakpointinabackingbean
ormodelmethodthatiscalledwithyourequest.IfyouwaitinthebreakpointforStuckMaxThreadTime,
younoticeaStuckThreadtracewillbeshowninserverslog:
<16=201112:28:22EET><Error><WebLogicServer><BEA000337><[STUCK]ExecuteThread:'2'forqueue:
'weblogic.kernel.Default(selftuning)'hasbeenbusyfor"134"secondsworkingonthe
request"weblogic.servlet.internal.ServletRequestImpl@6e6f4718[
GET/////days.xhtmlHTTP/1.1
Connection:keepalive
UserAgent:Mozilla/5.0(WindowsNT6.1;WOW64)AppleWebKit/535.2(KHTML,likeGecko)Chrome/15.0.874.120Safa
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
AcceptEncoding:gzip,deflate,sdch
AcceptLanguage:enGB,enUS;q=0.8,en;q=0.6
 AcceptCharset:ISO88591,utf8;q=0.7,*;q=0.3
Cookie:JSESSIONID=DYG5TDTZSnKLTFw5CMMdLCD9sPsZS4Jqlmxj9wdGNyt1BnPcfNrR!1520792836


]",whichismorethantheconfiguredtime(StuckThreadMaxTime)of"60"seconds.Stacktrace:
(.java:83)
javax.faces.component.UIComponentBase.encodeBegin(UIComponentBase.java:823)
com.sun.faces.renderkit.html_basic.HtmlBasicRenderer.encodeRecursive(HtmlBasicRenderer.java:285)
com.sun.faces.renderkit.html_basic.GridRenderer.renderRow(GridRenderer.java:185)
com.sun.faces.renderkit.html_basic.GridRenderer.encodeChildren(GridRenderer.java:129)
javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:848)
org.primefaces.renderkit.CoreRenderer.renderChild(CoreRenderer.java:55)
org.primefaces.renderkit.CoreRenderer.renderChildren(CoreRenderer.java:43)
org.primefaces.component.fieldset.FieldsetRenderer.encodeContent(FieldsetRenderer.java:95)
org.primefaces.component.fieldset.FieldsetRenderer.encodeMarkup(FieldsetRenderer.java:76)
org.primefaces.component.fieldset.FieldsetRenderer.encodeEnd(FieldsetRenderer.java:53)
javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:878)
javax.faces.component.UIComponent.encodeAll(UIComponent.java:1620)
javax.faces.render.Renderer.encodeChildren(Renderer.java:168)
javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:848)
org.primefaces.renderkit.CoreRenderer.renderChild(CoreRenderer.java:55)
org.primefaces.renderkit.CoreRenderer.renderChildren(CoreRenderer.java:43)
org.primefaces.component.panel.PanelRenderer.encodeContent(PanelRenderer.java:229)
org.primefaces.component.panel.PanelRenderer.encodeMarkup(PanelRenderer.java:152)

Moredigging:

ExcellentpostbyFrankMunz:WebLogicStuckThreads:Creating,UnderstandingandDealing
withthem.UpdatedforWeblogic12c.IncludessampleappforcreatingStuckThreadtoo.
http://stackoverflow.com/questions/2709410/weblogicstuckthreadprotection

src:

MaxenceButtonexcellentpost:http://mbutton.blogspot.com/2008/07/usingwlsttoperform
regular.html
http://download.oracle.com/docs/cd/E13222_01/wls/docs81/perform/WLSTuning.html#112571
4
http://download.oracle.com/docs/cd/E21764_01/web.1111/e13701/overload.htm
http://java.syscon.com/node/358060?page=0,0

UnleashthepowerofyourAPIswithfutureproofAPImanagementCreateyour
accountandstartyourfreetrialtoday,broughttoyouinpartnershipwith3scale.

LikeThisArticle?ReadMoreFromDZone
DeploymentAutomationofDocker YourOpinionWantedWhatWould
WebLogicClusteronAnyCloud YouLiketoSeeinOTNVirtual
 WebLogicClusteronAnyCloud YouLiketoSeeinOTNVirtual
Summits?

SolvingaProducerConsumer FreeDZoneRefcard
ProbleminJava RESTfulAPILifecycleManagement

Topics:JAVA, WEBLOGIC, HTTP

PublishedatDZonewithpermissionofSpyrosDoulgeridis,DZoneMVB.Seetheoriginalarticlehere.
OpinionsexpressedbyDZonecontributorsaretheirown.