JavaConcurrency
InPractice
BrianGöetz
TimPeierls
JoshuaBloch
JosephBowbeer
DavidHolmes
DougLea
AddisonͲWesleyProfessional
ISBNͲ10:0Ͳ321Ͳ34960Ͳ1
ISBNͲ13:978Ͳ0Ͳ321Ͳ34960Ͳ6
ii JavaConcurrencyInPractice
Index
Index
ii
Preface
xiii
Chapter 1 - Introduction
1
1.1. A (Very) Brief History of Concurrency
2
1.2. Benefits of Threads
3
1.3. Risks of Threads
5
1.4. Threads are Everywhere
8
Part I: Fundamentals
10
Chapter 2. Thread Safety
11
Chapter 3. Sharing Objects
23
Chapter 4. Composing Objects
37
Chapter 5. Building Blocks
51
<Index
iii
Part II: Structuring Concurrent Applications
71
Chapter 6. Task Execution
72
Chapter 7. Cancellation and Shutdown
85
Chapter 8. Applying Thread Pools
104
Chapter 9. GUI Applications
117
Part III: Liveness, Performance, and Testing
127
Chapter 10. Avoiding Liveness Hazards
128
Chapter 11. Performance and Scalability
137
Chapter 12. Testing Concurrent Programs
153
Part IV: Advanced Topics
170
Chapter 13 - Explicit Locks
171
iv JavaConcurrencyInPractice
Chapter 14 - Building Custom Synchronizers
179
Chapter 15. Atomic Variables and Non-blocking Synchronization
195
Chapter 16. The Java Memory Model
207
Appendix A. Annotations for Concurrency
216
Bibliography
217
1BListingandImageIndex
v
ListingandImageIndex
Preface
v
Listing1.BadWaytoSortaList.
xiv
Listing2.LessthanOptimalWaytoSortaList.
xv
Chapter 1. Introduction
11
Listing1.1.NonͲthreadͲsafeSequenceGenerator.
5
Figure1.1.UnluckyExecutionofUnsafeSequence.Nextvalue.
5
Listing1.2.ThreadͲsafeSequenceGenerator.
6
Chapter 2. Thread Safety
11
Listing2.1.AStatelessServlet.
13
Listing2.2.ServletthatCountsRequestswithouttheNecessarySynchronization.
14
Listing2.3.RaceConditioninLazyInitialization.
15
Listing2.4.ServletthatCountsRequestsUsingAtomicLong.
16
Listing2.5.ServletthatAttemptstoCacheitsLastResultwithoutAdequateAtomicity.
17
Listing2.6.ServletthatCachesLastResult,ButwithUnacceptablyPoorConcurrency.
18
Listing2.7.CodethatwouldDeadlockifIntrinsicLockswereNotReentrant.
18
Figure2.1.PoorConcurrencyofSynchronizedFactorizer.
21
Listing2.8.ServletthatCachesitsLastRequestandResult.
21
Chapter 3. Sharing Objects
23
Listing3.1.SharingVariableswithoutSynchronization.
23
Listing3.2.NonͲthreadͲsafeMutableIntegerHolder.
24
Listing3.3.ThreadͲsafeMutableIntegerHolder.
24
Figure3.1.VisibilityGuaranteesforSynchronization.
25
Listing3.4.CountingSheep.
26
Listing3.5.PublishinganObject.
27
Listing3.6.AllowingInternalMutableStatetoEscape.
27
Listing3.7.ImplicitlyAllowingthethisReferencetoEscape.
28
Listing3.8.UsingaFactoryMethodtoPreventthethisReferencefromEscapingDuringConstruction.
28
Listing3.9.ThreadConfinementofLocalPrimitiveandReferenceVariables.
30
Listing3.10.UsingThreadLocaltoEnsurethreadConfinement.
30
Listing3.11.ImmutableClassBuiltOutofMutableUnderlyingObjects.
32
Listing3.12.ImmutableHolderforCachingaNumberanditsFactors.
33
Listing3.13.CachingtheLastResultUsingaVolatileReferencetoanImmutableHolderObject.
33
Listing3.14.PublishinganObjectwithoutAdequateSynchronization.
33
vi JavaConcurrencyInPractice
Listing3.15.ClassatRiskofFailureifNotProperlyPublished.
34
Chapter 4. Composing Objects
37
Listing4.1.SimpleThreadͲsafeCounterUsingtheJavaMonitorPattern.
37
Listing4.2.UsingConfinementtoEnsureThreadSafety.
39
Listing4.3.GuardingStatewithaPrivateLock.
40
Listing4.4.MonitorͲbasedVehicleTrackerImplementation.
42
Listing4.5.MutablePointClassSimilartoJava.awt.Point.
42
Listing4.6.ImmutablePointclassusedbyDelegatingVehicleTracker.
42
Listing4.7.DelegatingThreadSafetytoaConcurrentHashMap.
43
Listing4.8.ReturningaStaticCopyoftheLocationSetInsteadofa"Live"One.
43
Listing4.9.DelegatingThreadSafetytoMultipleUnderlyingStateVariables.
44
Listing4.10.NumberRangeClassthatdoesNotSufficientlyProtectItsInvariants.
45
Listing4.11.ThreadͲsafeMutablePointClass.
46
Listing4.12.VehicleTrackerthatSafelyPublishesUnderlyingState.
46
Listing4.13.ExtendingVectortohaveaPutͲifͲabsentMethod.
47
Listing4.14.NonͲthreadͲsafeAttempttoImplementPutͲifͲabsent.
48
Listing4.15.ImplementingPutͲifͲabsentwithClientͲsideLocking.
48
Listing4.16.ImplementingPutͲifͲabsentUsingComposition.
49
Chapter 5. Building Blocks
51
Listing5.1.CompoundActionsonaVectorthatmayProduceConfusingResults.
51
Figure5.1.InterleavingofGetlastandDeletelastthatthrows
ArrayIndexOutOfBoundsException.
51
Listing5.2.CompoundActionsonVectorUsingClientͲsideLocking.
52
Listing5.3.IterationthatmayThrowArrayIndexOutOfBoundsException.
52
Listing5.4.IterationwithClientͲsideLocking.
52
Listing5.5.IteratingaListwithanIterator.
53
Listing5.6.IterationHiddenwithinStringConcatenation.
54
Listing5.7.ConcurrentMapInterface.
56
Listing5.8.ProducerandConsumerTasksinaDesktopSearchApplication.
58
Listing5.9.StartingtheDesktopSearch.
58
Listing5.10.RestoringtheInterruptedStatussoasNottoSwallowtheInterrupt.
60
Listing5.11.UsingCountDownLatchforStartingandStoppingThreadsinTimingTests.
61
Listing5.12.UsingFutureTasktoPreloadDatathatisNeededLater.
62
Listing5.13.CoercinganUncheckedThrowabletoaRuntimeException.
62
Listing5.14.UsingSemaphoretoBoundaCollection.
64
1BListingandImageIndex
vii
Listing5.15.CoordinatingComputationinaCellularAutomatonwith
64
Listing5.16.InitialCacheAttemptUsingHashMapandSynchronization.
66
Figure5.2.PoorConcurrencyof
66
Figure5.3.TwoThreadsComputingtheSameValueWhenUsing
67
Listing5.17.ReplacingHashMapwithConcurrentHashMap.
67
Figure5.4.UnluckyTimingthatcouldCauseMemorizer3toCalculatetheSameValueTwice.
68
Listing5.18.MemorizingWrapperUsingFutureTask.
68
Listing5.19.FinalImplementationofMemorizer.
69
Listing5.20.FactorizingServletthatCachesResultsUsingMemorizer.
69
Chapter 6. Task Execution
72
Listing6.1.SequentialWebServer.
72
Listing6.2.WebServerthatStartsaNewThreadforEachRequest.
73
Listing6.3.ExecutorInterface.
74
Listing6.4.WebServerUsingaThreadPool.
75
Listing6.5.ExecutorthatStartsaNewThreadforEachTask.
75
Listing6.6.ExecutorthatExecutesTasksSynchronouslyintheCallingThread.
75
Listing6.7.LifecycleMethodsinExecutorService.
77
Listing6.8.WebServerwithShutdownSupport.
77
Listing6.9.ClassIllustratingConfusingTimerBehavior.
78
Listing6.10.RenderingPageElementsSequentially.
79
Listing6.11.CallableandFutureInterfaces.
80
Listing6.12.DefaultImplementationofnewTaskForinThreadPoolExecutor.
80
Listing6.13.WaitingforImageDownloadwithFuture.
81
Listing6.14.QueueingFutureClassUsedByExecutorCompletionService.
82
Listing6.15.UsingCompletionServicetoRenderPageElementsastheyBecomeAvailable.
82
Listing6.16.FetchinganAdvertisementwithaTimeBudget.
83
Listing6.17.RequestingTravelQuotesUnderaTimeBudget.
84
Chapter 7. Cancellation and Shutdown
85
Listing7.1.UsingaVolatileFieldtoHoldCancellationState.
86
Listing7.2.GeneratingaSecond'sWorthofPrimeNumbers.
86
Listing7.3.UnreliableCancellationthatcanLeaveProducersStuckinaBlockingOperation.
87
Listing7.4.InterruptionMethodsinThread.
87
Listing7.5.UsingInterruptionforCancellation.
88
Listing7.6.PropagatingInterruptedExceptiontoCallers.
89
Listing7.7.NonͲcancelableTaskthatRestoresInterruptionBeforeExit.
90
Listing7.8.SchedulinganInterruptonaBorrowedThread.
90
viii JavaConcurrencyInPractice
Listing7.9.InterruptingaTaskinaDedicatedThread.
91
Listing7.10.CancellingaTaskUsingFuture.
92
Listing7.11.EncapsulatingNonstandardCancellationinaThreadbyOverridingInterrupt.
93
Listing7.12.EncapsulatingNonstandardCancellationinaTaskwithNewtaskfor.
94
Listing7.13.ProducerͲConsumerLoggingServicewithNoShutdownSupport.
95
Listing7.14.UnreliableWaytoAddShutdownSupporttotheLoggingService.
95
Listing7.15.AddingReliableCancellationtoLogWriter.
96
Listing7.16.LoggingServicethatUsesanExecutorService.
97
Listing7.17.ShutdownwithPoisonPill.
97
Listing7.18.ProducerThreadforIndexingService.
98
Listing7.19.ConsumerThreadforIndexingService.
98
Listing7.20.UsingaPrivateExecutorWhoseLifetimeisBoundedbyaMethodCall.
98
Listing7.21.ExecutorServicethatKeepsTrackofCancelledTasksAfterShutdown.
99
Listing7.22.UsingTRackingExecutorServicetoSaveUnfinishedTasksforLaterExecution.
100
Listing7.23.TypicalThreadͲpoolWorkerThreadStructure.
101
Listing7.24.UncaughtExceptionHandlerInterface.
101
Listing7.25.UncaughtExceptionHandlerthatLogstheException.
101
Listing7.26.RegisteringaShutdownHooktoStoptheLoggingService.
103
Chapter 8. Applying Thread Pools
104
Listing8.1.TaskthatDeadlocksinaSingleͲthreadedExecutor.
105
Listing8.2.GeneralConstructorforThreadPoolExecutor.
107
Listing8.3.CreatingaFixedͲsizedThreadPoolwithaBoundedQueueandtheCallerͲrunsSaturationPolicy. 109
Listing8.4.UsingaSemaphoretoThrottleTaskSubmission.
109
Listing8.5.ThreadFactoryInterface.
109
Listing8.6.CustomThreadFactory.
110
Listing8.7.CustomThreadBaseClass.
111
Listing8.8.ModifyinganExecutorCreatedwiththeStandardFactories.
111
Listing8.9.ThreadPoolExtendedwithLoggingandTiming.
112
Listing8.10.TransformingSequentialExecutionintoParallelExecution.
112
Listing8.11.TransformingSequentialTailͲrecursionintoParallelizedRecursion.
113
Listing8.12.WaitingforResultstobeCalculatedinParallel.
113
Listing8.13.AbstractionforPuzzlesLikethe"SlidingBlocksPuzzle".
113
Listing8.14.LinkNodeforthePuzzleSolverFramework.
114
Listing8.15.SequentialPuzzleSolver.
115
Listing8.16.ConcurrentVersionofPuzzleSolver.
115
Listing8.17.ResultͲbearingLatchUsedbyConcurrentPuzzleSolver.
116
1BListingandImageIndex
ix
Listing8.18.SolverthatRecognizeswhenNoSolutionExists.
116
Chapter 9. GUI Applications
117
Figure9.1.ControlFlowofaSimpleButtonClick.
119
Listing9.1.ImplementingSwingUtilitiesUsinganExecutor.
120
Listing9.2.ExecutorBuiltAtopSwingUtilities.
120
Listing9.3.SimpleEventListener.
120
Figure9.2.ControlFlowwithSeparateModelandViewObjects.
121
Listing9.4.BindingaLongͲrunningTasktoaVisualComponent.
121
Listing9.5.LongͲrunningTaskwithUserFeedback.
122
Listing9.6.CancellingaLongͲrunningTask.
122
Listing9.7.BackgroundTaskClassSupportingCancellation,CompletionNotification,andProgressNotification.
124
Listing9.8.InitiatingaLongͲrunning,CancellableTaskwithBackgroundTask.
124
Chapter 10. Avoiding Liveness Hazards
128
Figure10.1.UnluckyTiminginLeftRightDeadlock.
128
Listing10.1.SimpleLockͲorderingDeadlock.
129
Listing10.2.DynamicLockͲorderingDeadlock.
129
Listing10.3.InducingaLockOrderingtoAvoidDeadlock.
130
Listing10.4.DriverLoopthatInducesDeadlockUnderTypicalConditions.
131
Listing10.5.LockͲorderingDeadlockBetweenCooperatingObjects.Don'tDothis.
132
Listing10.6.UsingOpenCallstoAvoidingDeadlockBetweenCooperatingObjects.
133
Listing10.7.PortionofThreadDumpAfterDeadlock.
135
Chapter 11. Performance and Scalability
137
Figure11.1.MaximumUtilizationUnderAmdahl'sLawforVariousSerializationPercentages.
140
Listing11.1.SerializedAccesstoaTaskQueue.
141
Figure11.2.ComparingQueueImplementations.
141
Listing11.2.SynchronizationthathasNoEffect.
142
Listing11.3.CandidateforLockElision.
143
Listing11.4.HoldingaLockLongerthanNecessary.
145
Listing11.5.ReducingLockDuration.
145
Listing11.6.CandidateforLockSplitting.
146
Listing11.7.ServerStatusRefactoredtoUseSplitLocks.
146
Listing11.8.HashͲbasedMapUsingLockStriping.
148
Figure11.3.ComparingScalabilityofMapImplementations.
150
Chapter 12. Testing Concurrent Programs
153
Listing12.1.BoundedBufferUsingSemaphore.
154
x JavaConcurrencyInPractice
Listing12.2.BasicUnitTestsforBoundedBuffer.
154
Listing12.3.TestingBlockingandResponsivenesstoInterruption.
156
Listing12.4.MediumͲqualityRandomNumberGeneratorSuitableforTesting.
157
Listing12.5.ProducerͲconsumerTestProgramforBoundedBuffer.
158
Listing12.6.ProducerandConsumerClassesUsedinPutTakeTest.
158
Listing12.7.TestingforResourceLeaks.
159
Listing12.8.ThreadFactoryforTestingThreadPoolExecutor.
160
Listing12.9.TestMethodtoVerifyThreadPoolExpansion.
160
Listing12.10.UsingThread.yieldtoGenerateMoreInterleavings.
160
Listing12.11.BarrierͲbasedTimer.
161
Figure12.1.TimedPutTakeTestwithVariousBufferCapacities.
162
Listing12.12.TestingwithaBarrierͲbasedTimer.
162
Listing12.13.DriverProgramͲforTimedPutTakeTest.
163
Figure12.2.ComparingBlockingQueueImplementations.
163
Figure12.3.CompletionTimeHistogramforTimedPutTakeTestwithDefault(NonͲfair)andFair
Semaphores.
164
Figure12.4.CompletionTimeHistogramforTimedPutTakeTestwithSingleͲitemBuffers.
164
Figure12.5.ResultsBiasedbyDynamicCompilation.
165
Chapter 13 - Explicit Locks
171
Listing13.1.LockInterface.
171
Listing13.2.GuardingObjectStateUsingReentrantLock.
171
Listing13.3.AvoidingLockͲorderingDeadlockUsingtrylock.
173
Listing13.4.LockingwithaTimeBudget.
173
Listing13.5.InterruptibleLockAcquisition.
173
Figure13.1.IntrinsicLockingVersusReentrantLockPerformanceonJava5.0andJava6.
174
Figure13.2.FairVersusNonͲfairLockPerformance.
175
Listing13.6.ReadWriteLockInterface.
176
Listing13.7.WrappingaMapwithaReadͲwriteLock.
178
Figure13.3.ReadͲwriteLockPerformance.
178
Chapter 14 - Building Custom Synchronizers
179
Listing14.1.StructureofBlockingStateͲdependentActions.
179
Listing14.2.BaseClassforBoundedBufferImplementations.
180
Listing14.3.BoundedBufferthatBalksWhenPreconditionsareNotMet.
180
Listing14.4.ClientLogicforCallingGrumpyBoundedBuffer.
181
Figure14.1.ThreadOversleepingBecausetheConditionBecameTrueJustAfterItWenttoSleep.
181
Listing14.5.BoundedBufferUsingCrudeBlocking.
182
1BListingandImageIndex
xi
Listing14.6.BoundedBufferUsingConditionQueues.
183
Listing14.7.CanonicalFormforStateͲdependentMethods.
184
Listing14.8.UsingConditionalNotificationinBoundedBuffer.put.
186
Listing14.9.RecloseableGateUsingWaitandNotifyall.
187
Listing14.10.ConditionInterface.
188
Listing14.11.BoundedBufferUsingExplicitConditionVariables.
189
Listing14.12.CountingSemaphoreImplementedUsingLock.
190
Listing14.13.CanonicalFormsforAcquisitionandReleaseinAQS.
191
Listing14.14.BinaryLatchUsingAbstractQueuedSynchronizer.
192
Listing14.15.tryAcquireImplementationFromNonͲfairReentrantLock.
193
Listing14.16.tryacquiresharedandtryreleasesharedfromSemaphore.
193
Chapter 15. Atomic Variables and Non-blocking Synchronization
195
Listing15.1.SimulatedCASOperation.
197
Listing15.2.NonͲblockingCounterUsingCAS.
197
Listing15.3.PreservingMultivariableInvariantsUsingCAS.
199
Figure15.1.LockandAtomicIntegerPerformanceUnderHighContention.
200
Figure15.2.LockandAtomicIntegerPerformanceUnderModerateContention.
200
Listing15.4.RandomNumberGeneratorUsingReentrantLock.
200
Listing15.5.RandomNumberGeneratorUsingAtomicInteger.
201
Listing15.6.NonͲblockingStackUsingTreiber'sAlgorithm(Treiber,1986).
203
Figure15.3.QueuewithTwoElementsinQuiescentState.
203
Figure15.4.QueueinIntermediateStateDuringInsertion.
204
Figure15.5.QueueAgaininQuiescentStateAfterInsertionisComplete.
204
Listing15.7.InsertionintheMichaelͲScottNonͲblockingQueueAlgorithm(MichaelandScott,1996).
205
Listing15.8.UsingAtomicFieldUpdatersinConcurrentLinkedQueue.
205
Chapter 16. The Java Memory Model
207
Figure16.1.InterleavingShowingReorderinginPossibleReordering.
208
Listing16.1.InsufficientlySynchronizedProgramthatcanhaveSurprisingResults.
209
Figure16.2.IllustrationofHappensͲbeforeintheJavaMemoryModel.
210
Listing16.2.InnerClassofFutureTaskIllustratingSynchronizationPiggybacking.
211
Listing16.3.UnsafeLazyInitialization.Don'tDothis.
212
Listing16.4.ThreadͲsafeLazyInitialization.
213
Listing16.5.EagerInitialization.
213
Listing16.6.LazyInitializationHolderClassIdiom.
213
Listing16.7.DoubleͲcheckedͲlockingAntiͲpattern.
214
Listing16.8.InitializationSafetyforImmutableObjects.
215
xii JavaConcurrencyInPractice
2BPreface
xiii
Preface
At this writing, multiͲcore processors are just now becoming inexpensive enough for midrange desktop systems. Not coincidentally,manydevelopmentteamsarenoticingmoreandmorethreadingͲrelatedbugreportsintheirprojects.In a recent post on the NetBeans developer site, one of the core maintainers observed that a single class had been patchedover14timestofixthreadingͲrelatedproblems.DionAlmaer,formereditorofTheServerSide,recentlyblogged (after a painful debugging session that ultimately revealed a threading bug) that most Java programs are so rife with concurrencybugsthattheyworkonly"byaccident".
Indeed,developing,testinganddebuggingmultithreadedprogramscanbeextremelydifficultbecauseconcurrencybugs donotmanifestthemselvespredictably.Andwhentheydosurface,itisoftenattheworstpossibletimeinproduction, underheavyload.
One of the challenges of developing concurrent programs in Java is the mismatch between the concurrency features offeredbytheplatformandhowdevelopersneedtothinkaboutconcurrencyintheirprograms.Thelanguageprovides lowͲlevelmechanismssuchassynchronizationandconditionwaits,butthesemechanismsmustbeusedconsistentlyto implement applicationͲlevel protocols or policies. Without such policies, it is all too easy to create programs that compileandappeartoworkbutareneverthelessbroken.Manyotherwiseexcellentbooksonconcurrencyfallshortof theirgoalbyfocusingexcessivelyonlowͲlevelmechanismsandAPIsratherthandesignͲlevelpoliciesandpatterns.
Java 5.0 is a huge step forward for the development of concurrent applications in Java, providing new higherͲlevel componentsandadditionallowͲlevelmechanismsthatmakeiteasierfornovicesandexpertsaliketobuildconcurrent applications.TheauthorsaretheprimarymembersoftheJCPExpertGroupthatcreatedthesefacilities;inadditionto describingtheirbehaviorandfeatures,wepresenttheunderlyingdesignpatternsandanticipatedusagescenariosthat motivatedtheirinclusionintheplatformlibraries.
Ourgoalistogivereadersasetofdesignrulesandmentalmodelsthatmakeiteasierandmorefuntobuildcorrect, performantconcurrentclassesandapplicationsinJava.
WehopeyouenjoyJavaConcurrencyinPractice.
BrianGoetz
Williston,VT
March2006
HowtoUsethisBook
ToaddresstheabstractionmismatchbetweenJava'slowͲlevelmechanismsandthenecessarydesignͲlevelpolicies,we presentasimplifiedsetofrulesforwritingconcurrentprograms.Expertsmaylookattheserulesandsay"Hmm,that's notentirelytrue:classCisthreadͲsafeeventhoughitviolatesruleR."Whileitispossibletowritecorrectprogramsthat break our rules, doing so requires a deep understanding of the lowͲlevel details of the Java Memory Model, and we wantdeveloperstobeabletowritecorrectconcurrentprogramswithouthavingtomasterthesedetails.Consistently followingoursimplifiedruleswillproducecorrectandmaintainableconcurrentprograms.
We assume the reader already has some familiarity with the basic mechanisms for concurrency in Java. Java Concurrency in Practice is not an introduction to concurrency for that, see the threading chapter of any decent introductoryvolume,suchasTheJavaProgrammingLanguage(Arnoldetal.,2005).Norisitanencyclopedicreference forAllThingsConcurrencyforthat,seeConcurrentProgramminginJava(Lea,2000).Rather,itofferspracticaldesign rulestoassistdevelopersinthedifficultprocessofcreatingsafeandperformantconcurrentclasses.Whereappropriate, we crossͲreference relevant sections of The Java Programming Language, Concurrent Programming in Java, The Java LanguageSpecification(Goslingetal.,2005),andEffectiveJava(Bloch,2001)usingtheconventions[JPLn.m],[CPJn.m],
[JLSn.m],and[EJItemn].
Aftertheintroduction(Chapter1),thebookisdividedintofourparts:
Fundamentals. Part I (Chapters 2Ͳ5) focuses on the basic concepts of concurrency and thread safety, and how to compose threadͲsafe classes out of the concurrent building blocks provided by the class library. A "cheat sheet"
summarizingthemostimportantoftherulespresentedinPartIappearsonpage110.
xiv JavaConcurrencyInPractice
Chapters2(ThreadSafety)and3(SharingObjects)formthefoundationforthebook.Nearlyalloftherulesonavoiding concurrency hazards, constructing threadͲsafe classes, and verifying thread safety are here. Readers who prefer
"practice"to"theory"maybetemptedtoskipaheadtoPartII,butmakesuretocomebackandreadChapters2and3
beforewritinganyconcurrentcode!
Chapter 4 (Composing Objects) covers techniques for composing threadͲsafe classes into larger threadͲsafe classes.
Chapter5(BuildingBlocks)coverstheconcurrentbuildingblocksͲthreadͲsafecollectionsandsynchronizersͲprovided bytheplatformlibraries.
StructuringConcurrentApplications.PartII(Chapters6Ͳ9)describeshowtoexploitthreadstoimprovethethroughput or responsiveness of concurrent applications. Chapter 6 (Task Execution) covers identifying parallelizable tasks and executingthemwithinthetaskͲexecutionframework.Chapter7(CancellationandShutdown)dealswithtechniquesfor convincingtasksandthreadstoterminatebeforetheywouldnormallydoso;howprogramsdealwithcancellationand shutdownisoftenoneofthefactorsthatseparatetrulyrobustconcurrentapplicationsfromthosethatmerelywork.
Chapter8(ApplyingThreadPools)addressessomeofthemoreadvancedfeaturesofthetaskͲexecutionframework.
Chapter9(GUIApplications)focusesontechniquesforimprovingresponsivenessinsingleͲthreadedsubsystems.
Liveness, Performance, and Testing. Part III (Chapters 10Ͳ12) concerns itself with ensuring that concurrent programs actuallydowhatyouwantthemtodoanddosowithacceptableperformance.Chapter10(AvoidingLivenessHazards) describes how to avoid liveness failures that can prevent programs from making forward progress. Chapter 11
(Performance and Scalability) covers techniques for improving the performance and scalability of concurrent code.
Chapter 12 (Testing Concurrent Programs) covers techniques for testing concurrent code for both correctness and performance.
AdvancedTopics.PartIV(Chapters13Ͳ16)coverstopicsthatarelikelytobeofinterestonlytoexperienceddevelopers: explicitlocks,atomicvariables,nonͲblockingalgorithms,anddevelopingcustomsynchronizers.
CodeExamples
WhilemanyofthegeneralconceptsinthisbookareapplicabletoversionsofJavapriortoJava5.0andeventononͲJava environments,mostofthecodeexamples(andallthestatementsabouttheJavaMemoryModel)assumeJava5.0or later.SomeofthecodeexamplesmayuselibraryfeaturesaddedinJava6.
Thecodeexampleshavebeencompressedtoreducetheirsizeandtohighlighttherelevantportions.Thefullversions of the code examples, as well as supplementary examples and errata, are available from the book's website, http://www.javaconcurrencyinpractice.com.
Thecodeexamplesareofthreesorts:"good"examples,"notsogood"examples,and"bad"examples.Goodexamples illustrate techniques that should be emulated. Bad examples illustrate techniques that should definitely not be emulated,andareidentifiedwitha"Mr.Yuk"icon[1]tomakeitclearthatthisis"toxic"code(seeListing1).NotͲsoͲgood examplesillustratetechniquesthatarenotnecessarilywrongbutarefragile,risky,orperformpoorly,andaredecorated witha"Mr.CouldBeHappier"iconasinListing2.
[1]Mr.YukisaregisteredtrademarkoftheChildren'sHospitalofPittsburghandappearsbypermission.
Listing1.BadWaytoSortaList.
public <T extends Comparable<? super T>> void sort(List<T> list) {
// Never returns the wrong answer!
System.exit(0);
}
Somereadersmayquestiontheroleofthe"bad"examplesinthisbook;afterall,abookshouldshowhowtodothings right, not wrong. The bad examples have two purposes. They illustrate common pitfalls, but more importantly they demonstrate how to analyze a program for thread safety Ͳ and the best way to do that is to see the ways in which threadsafetyiscompromised.
2BPreface
xv
Listing2.LessthanOptimalWaytoSortaList.
public <T extends Comparable<? super T>> void sort(List<T> list) {
for (int i=0; i<1000000; i++)
doNothing();
Collections.sort(list);
}
Acknowledgments
Thisbookgrewoutofthedevelopmentprocessforthejava.util.concurrentpackagethatwascreatedbytheJava CommunityProcessJSR166forinclusioninJava5.0.ManyotherscontributedtoJSR166;inparticularwethankMartin Buchholz for doing all the work related to getting the code into the JDK, and all the readers of the concurrency-interestmailinglistwhoofferedtheirsuggestionsandfeedbackonthedraftAPIs.
Thisbookhasbeentremendouslyimprovedbythesuggestionsandassistanceofasmallarmyofreviewers,advisors, cheerleaders,andarmchaircritics.WewouldliketothankDionAlmaer,TracyBialik,CindyBloch,MartinBuchholz,Paul Christmann,CliffClick,StuartHalloway,DavidHovemeyer,JasonHunter,MichaelHunter,JeremyHylton,HeinzKabutz, Robert Kuhar, Ramnivas Laddad, Jared Levy, Nicole Lewis, Victor Luchangco, Jeremy Manson, Paul Martin, Berna Massingill,MichaelMaurer,TedNeward,KirkPepperdine,BillPugh,SamPullara,RussRufer,BillScherer,JeffreySiegal, Bruce Tate, Gil Tene, Paul Tyma, and members of the Silicon Valley Patterns Group who, through many interesting technicalconversations,offeredguidanceandmadesuggestionsthathelpedmakethisbookbetter.
WeareespeciallygratefultoCliffBiffle,BarryHayes,DawidKurzyniec,AngelikaLanger,DoronRajwan,andBillVenners, whoreviewedtheentiremanuscriptinexcruciatingdetail,foundbugsinthecodeexamples,andsuggestednumerous improvements.
WethankKatrinaAveryforagreatcopyͲeditingjobandRosemarySimpsonforproducingtheindexunderunreasonable timepressure.WethankAmiDewarfordoingtheillustrations.
ThankstothewholeteamatAddisonͲWesleywhohelpedmakethisbookareality.AnnSellersgottheprojectlaunched andGregDoenchshepherdedittoasmoothcompletion;ElizabethRyanguideditthroughtheproductionprocess.
Wewouldalsoliketothankthethousandsofsoftwareengineerswhocontributedindirectlybycreatingthesoftware usedto createthis book,includingTEX,LATEX, AdobeAcrobat, pic,grap,AdobeIllustrator,Perl,ApacheAnt,IntelliJ
IDEA,GNUemacs,Subversion,TortoiseSVN,andofcourse,theJavaplatformandclasslibraries.
3BChapter1Ͳ Introduction Ͳ 10B1.1.A(Very)BriefHistoryofConcurrency 1
Chapter1ǦIntroduction
Writingcorrectprogramsishard;writingcorrectconcurrentprogramsisharder.Therearesimplymorethingsthatcan gowronginaconcurrentprogramthaninasequentialone.So,whydowebotherwithconcurrency?Threadsarean inescapable feature of the Java language, and they can simplify the development of complex systems by turning complicated asynchronous code into simpler straightͲline code. In addition, threads are the easiest way to tap the computingpowerofmultiprocessorsystems.And,asprocessorcountsincrease,exploitingconcurrencyeffectivelywill onlybecomemoreimportant.
2 JavaConcurrencyInPractice
1.1.A(Very)BriefHistoryofConcurrency
Intheancientpast,computersdidn'thaveoperatingsystems;theyexecutedasingleprogramfrombeginningtoend, andthatprogramhaddirectaccesstoalltheresourcesofthemachine.Notonlywasitdifficulttowriteprogramsthat ran on the bare metal, but running only a single program at a time was an inefficient use of expensive and scarce computerresources.
Operatingsystemsevolvedtoallowmorethanoneprogramtorunatonce,runningindividualprogramsinprocesses: isolated, independently executing programs to which the operating system allocates resources such as memory, file handles,andsecuritycredentials.Iftheyneededto,processescouldcommunicatewithoneanotherthroughavariety ofcoarseͲgrainedcommunicationmechanisms:sockets,signalhandlers,sharedmemory,semaphores,andfiles.
Several motivating factors led to the development of operating systems that allowed multiple programs to execute simultaneously:
Resource utilization. Programs sometimes have to wait for external operations such as input or output, and while waitingcandonousefulwork.Itismoreefficienttousethatwaittimetoletanotherprogramrun.
Fairness.Multipleusersandprogramsmayhaveequalclaimsonthemachine'sresources.Itispreferabletoletthem sharethecomputerviafinerͲgrainedtimeslicingthantoletoneprogramruntocompletionandthenstartanother.
Convenience. It is often easier or more desirable to write several programsthat each perform a single task and have themcoordinatewitheachotherasnecessarythantowriteasingleprogramthatperformsallthetasks.
Inearlytimesharingsystems,eachprocesswasavirtualvonNeumanncomputer;ithadamemoryspacestoringboth instructions and data, executing instructions sequentially according to the semantics of the machine language, and interacting with the outside world via the operating system through a set of I/O primitives. For each instruction executedtherewasaclearlydefined"nextinstruction",andcontrolflowedthroughtheprogramaccordingtotherules oftheinstructionset.Nearlyallwidelyusedprogramminglanguagestodayfollowthissequentialprogrammingmodel, wherethelanguagespecificationclearlydefines"whatcomesnext"afteragivenactionisexecuted.
Thesequentialprogrammingmodelisintuitiveandnatural,asitmodelsthewayhumanswork:doonethingatatime, in sequence mostly. Get out of bed, put on your bathrobe, go downstairs and start the tea. As in programming languages, each of these realͲworld actions is an abstraction for a sequence of finerͲgrained actions Ͳ open the cupboard,selectaflavoroftea,measuresometeaintothepot,seeifthere'senoughwaterintheteakettle,ifnotput somemorewaterin,setitonthestove,turnthestoveon,waitforthewatertoboil,andsoon.ThislaststepͲwaiting forthewatertoboilͲalsoinvolvesadegreeofasynchrony.Whilethewaterisheating,youhaveachoiceofwhattodoͲ
just wait, or do other tasks in that time such as starting the toast (another asynchronous task) or fetching the newspaper, while remaining aware that your attention will soon be needed by the teakettle. The manufacturers of teakettlesandtoastersknowtheirproductsareoftenusedinanasynchronousmanner,sotheyraiseanaudiblesignal when they complete their task. Finding the right balance of sequentiality and asynchrony is often a characteristic of efficientpeopleͲandthesameistrueofprograms.
Thesameconcerns(resourceutilization,fairness,andconvenience)thatmotivatedthedevelopmentofprocessesalso motivated the development of threads. Threads allow multiple streams of program control flow to coexist within a process. They share processͲwide resources such as memory and file handles, but each thread has its own program counter,stack,andlocalvariables.Threadsalsoprovideanaturaldecompositionforexploitinghardwareparallelismon multiprocessorsystems;multiplethreadswithinthesameprogramcanbescheduledsimultaneouslyonmultipleCPUs.
Threadsaresometimescalledlightweightprocesses,andmostmodernoperatingsystemstreatthreads,notprocesses, as the basic units of scheduling. In the absence of explicit coordination, threads execute simultaneously and asynchronouslywithrespecttooneanother.Sincethreadssharethememoryaddressspaceoftheirowningprocess,all threadswithinaprocesshaveaccesstothesamevariablesandallocateobjectsfromthesameheap,whichallowsfinerͲ
grained data sharing than interͲprocess mechanisms. But without explicit synchronization to coordinate access to shareddata,athreadmaymodifyvariablesthatanotherthreadisinthemiddleofusing,withunpredictableresults.
3BChapter1Ͳ IntroductionͲ11B1.2.BenefitsofThreads 3
1.2.BenefitsofThreads
Whenusedproperly,threadscanreducedevelopmentandmaintenancecostsandimprovetheperformanceofcomplex applications.Threadsmakeiteasiertomodelhowhumansworkandinteract,byturningasynchronousworkflowsinto mostly sequential ones. They can also turn otherwise convoluted code into straightͲline code that is easier to write, read,andmaintain.
ThreadsareusefulinGUIapplicationsforimprovingtheresponsivenessoftheuserinterface,andinserverapplications for improving resource utilization and throughput. They also simplify the implementation of the JVM Ͳ the garbage collector usually runs in one or more dedicated threads. Most nontrivial Java applications rely to some degree on threadsfortheirorganization.
1.2.1.ExploitingMultipleProcessors
Multiprocessor systems used to be expensive and rare, found only in large data centers and scientific computing facilities.Todaytheyarecheapandplentiful;evenlowͲendserverandmidrangedesktopsystemsoftenhavemultiple processors.Thistrendwillonlyaccelerate;asitgetshardertoscaleupclockrates,processormanufacturerswillinstead put more processor cores on a single chip. All the major chip manufacturers have begun this transition, and we are alreadyseeingmachineswithdramaticallyhigherprocessorcounts.
Sincethebasicunitofschedulingisthethread,aprogramwithonlyonethreadcanrunonatmostoneprocessorata time.OnatwoͲprocessorsystem,asingleͲthreadedprogramisgivingupaccesstohalftheavailableCPUresources;ona 100Ͳprocessor system, it is giving up access to 99%. On the other hand, programs with multiple active threads can execute simultaneously on multiple processors. When properly designed, multithreaded programs can improve throughputbyutilizingavailableprocessorresourcesmoreeffectively.
Using multiple threads can also help achieve better throughput on singleͲprocessor systems. If a program is singleͲ
threaded, the processor remains idle while it waits for a synchronous I/O operation to complete. In a multithreaded program,anotherthreadcanstillrunwhilethefirstthreadiswaitingfortheI/Otocomplete,allowingtheapplicationto stillmakeprogressduringtheblockingI/O.(Thisislikereadingthenewspaperwhilewaitingforthewatertoboil,rather thanwaitingforthewatertoboilbeforestartingtoread.)
1.2.2.SimplicityofModeling
It is often easier to manage your time when you have only one type of task to perform (fix these twelve bugs) than when you have several (fix the bugs, interview replacement candidates for the system administrator, complete your team'sperformanceevaluations,andcreatetheslidesforyourpresentationnextweek).Whenyouhaveonlyonetype oftasktodo,youcanstartatthetopofthepileandkeepworkinguntilthepileisexhausted(oryouare);youdon't havetospendanymentalenergyfiguringoutwhattoworkonnext.Ontheotherhand,managingmultiplepriorities anddeadlinesandswitchingfromtasktotaskusuallycarriessomeoverhead.
The same is true for software: a program that processes one type of task sequentially is simpler to write, less errorͲ
prone,andeasiertotestthanonemanagingmultipledifferenttypesoftasksatonce.Assigningathreadtoeachtypeof taskortoeachelementinasimulationaffordstheillusionofsequentialityandinsulatesdomainlogicfromthedetailsof scheduling,interleavedoperations,asynchronousI/O,andresourcewaits.Acomplicated,asynchronousworkflowcan bedecomposedintoanumberofsimpler,synchronousworkflowseachrunninginaseparatethread,interactingonly witheachotheratspecificsynchronizationpoints.
This benefit is often exploited by frameworks such as servlets or RMI (Remote Method Invocation). The framework handles the details of request management, thread creation, and load balancing, dispatching portions of the request handling to the appropriate application component at the appropriate point in the workͲflow. Servlet writers do not needtoworryabouthowmanyotherrequestsarebeingprocessedatthesametimeorwhetherthesocketinputand output streams block; when a servlet's service method is called in response to a web request, it can process the requestsynchronouslyasifitwereasingleͲthreadedprogram.Thiscansimplifycomponentdevelopmentandreduce thelearningcurveforusingsuchframeworks.
1.2.3.SimplifiedHandlingofAsynchronousEvents
Aserverapplicationthatacceptssocketconnectionsfrommultipleremoteclientsmaybeeasiertodevelopwheneach connectionisallocateditsownthreadandallowedtousesynchronousI/O.
4 JavaConcurrencyInPractice
If an application goes to read from a socket when no data is available, read blocks until some data is available. In a singleͲthreadedapplication,thismeansthatnotonlydoesprocessingthecorrespondingrequeststall,butprocessingof all requests stalls while the single thread is blocked. To avoid this problem, singleͲthreaded server applications are forcedtousenonͲblockingI/O,whichisfarmorecomplicatedanderrorͲpronethansynchronousI/O.However,ifeach requesthasitsownthread,thenblockingdoesnotaffecttheprocessingofotherrequests.
Historically,operatingsystemsplacedrelativelylowlimitsonthenumberofthreadsthataprocesscouldcreate,asfew asseveralhundred(orevenless).Asaresult,operatingsystemsdevelopedefficientfacilitiesformultiplexedI/O,such as the Unix select and poll system calls, and to access these facilities, the Java class libraries acquired a set of packages (java.nio) for nonͲblocking I/O. However, operating system support for larger numbers of threads has improved significantly, making the threadͲperͲclient model practical even for large numbers of clients on some platforms.[1]
[1]TheNPTLthreadspackage,nowpartofmostLinuxdistributions,wasdesignedtosupporthundredsofthousandsofthreads.NonͲblockingI/O
hasitsownbenefits,butbetterOSsupportforthreadsmeansthattherearefewersituationsforwhichitisessential.
1.2.4.MoreResponsiveUserInterfaces
GUIapplicationsusedtobesingleͲthreaded,whichmeantthatyouhadtoeitherfrequentlypollthroughoutthecodefor input events (which is messy and intrusive) or execute all application code indirectly through a "main event loop". If codecalledfromthemaineventlooptakestoolongtoexecute,theuserinterfaceappearsto"freeze"untilthatcode finishes,becausesubsequentuserinterfaceeventscannotbeprocesseduntilcontrolisreturnedtothemaineventloop.
Modern GUI frameworks, such as the AWT and Swing toolkits, replace the main event loop with an event dispatch thread(EDT).Whenauserinterfaceeventsuchasabuttonpressoccurs,applicationͲdefinedeventhandlersarecalled in the event thread. Most GUI frameworks are singleͲthreaded subsystems, so the main event loop is effectively still present,butitrunsinitsownthreadunderthecontroloftheGUItoolkitratherthantheapplication.
IfonlyshortͲlivedtasksexecuteintheeventthread,theinterfaceremainsresponsivesincetheeventthreadisalways abletoprocessuseractionsreasonablyquickly.However,processingalongͲrunningtaskintheeventthread,suchas spellͲcheckingalargedocumentorfetchingaresourceoverthenetwork,impairsresponsiveness.Iftheuserperforms anactionwhilethistaskisrunning,thereisalongdelaybeforetheeventthreadcanprocessorevenacknowledgeit.To addinsulttoinjury,notonlydoestheUIbecomeunresponsive,butitisimpossibletocanceltheoffendingtaskevenif theUIprovidesacancelbuttonbecausetheeventthreadisbusyandcannothandlethecancelbuttonͲpresseventuntil the lengthy task completes! If, however, the longͲrunning task is instead executed in a separate thread, the event threadremainsfreetoprocessUIevents,makingtheUImoreresponsive.
3BChapter1Ͳ IntroductionͲ12B1.3.RisksofThreads
5
1.3.RisksofThreads
Java's builtͲin support for threads is a doubleͲedged sword. While it simplifies the development of concurrent applications by providing language and library support and a formal crossͲplatform memory model (it is this formal crossͲplatform memory model that makes possible the development of writeͲonce, runͲanywhere concurrent applicationsinJava),italsoraisesthebarfordevelopersbecausemoreprogramswillusethreads.Whenthreadswere more esoteric, concurrency was an "advanced" topic; now, mainstream developers must be aware of threadͲsafety issues.
1.3.1.SafetyHazards
Thread safety can be unexpectedly subtle because, in the absence of sufficient synchronization, the ordering of operations in multiple threads is unpredictable and sometimes surprising. UnsafeSequence in Listing 1.1, which is supposedtogenerateasequenceofuniqueintegervalues,offersasimpleillustrationofhowtheinterleavingofactions in multiple threads can lead to undesirable results. It behaves correctly in a singleͲthreaded environment, but in a multithreadedenvironmentdoesnot.
Listing1.1.NonǦthreadǦsafeSequenceGenerator.
@NotThreadSafe
public class UnsafeSequence {
private int value;
/** Returns a unique value. */
public int getNext() {
return value++;
}
}
TheproblemwithUnsafeSequenceisthatwithsomeunluckytiming,twothreadscouldcall getNextandreceive the samevalue.Figure1.1showshowthiscanhappen.Theincrementnotation,nextValue++,mayappeartobeasingle operation, but is in fact three separate operations: read the value, add one to it, and write out the new value. Since operationsinmultiplethreadsmaybearbitrarilyinterleavedbytheruntime,itispossiblefortwothreadstoreadthe valueat the sametime,bothsee thesamevalue,andthenbothaddonetoit.Theresult isthatthesamesequence numberisreturnedfrommultiplecallsindifferentthreads.
Figure1.1.UnluckyExecutionofUnsafeSequence.Nextvalue.
Diagrams like Figure 1.1 depict possible interleavings of operations in different threads. In these diagrams, time runs fromlefttoright,andeachlinerepresentstheactivitiesofadifferentthread.Theseinterleavingdiagramsusuallydepict theworstcase[2]andareintendedtoshowthedangerofincorrectlyassumingthingswillhappeninaparticularorder.
[2]Actually,aswe'llseeinChapter3,theworstcasecanbeevenworsethanthesediagramsusuallyshowbecauseofthepossibilityofreordering.
UnsafeSequence uses a nonstandard annotation: @NotThreadSafe. This is one of several custom annotations used throughoutthisbooktodocumentconcurrencypropertiesofclassesandclassmembers.(OtherclassͲlevelannotations usedinthiswayare@ThreadSafeand@Immutable;seeAppendixAfordetails.)Annotationsdocumentingthreadsafety are useful to multiple audiences. If a class is annotated with @ThreadSafe, users can use it with confidence in a multithreaded environment, maintainers are put on notice that it makes thread safety guarantees that must be preserved,andsoftwareanalysistoolscanidentifypossiblecodingerrors.
6 JavaConcurrencyInPractice
UnsafeSequenceillustratesacommonconcurrencyhazardcalledaracecondition.WhetherornotnextValuereturnsa unique value when called from multiple threads, as required by its specification, depends on how the runtime interleavestheoperationsͲwhichisnotadesirablestateofaffairs.
Becausethreadssharethesamememoryaddressspaceandrunconcurrently,theycanaccessormodifyvariablesthat otherthreadsmightbeusing.Thisisatremendousconvenience,becauseitmakesdatasharingmucheasierthanwould otherinterͲthreadcommunicationsmechanisms.Butitisalsoasignificantrisk:threadscanbeconfusedbyhavingdata changeunexpectedly.AllowingmultiplethreadstoaccessandmodifythesamevariablesintroducesanelementofnonͲ
sequentialityintoanotherwisesequentialprogrammingmodel,whichcanbeconfusinganddifficulttoreasonabout.
Foramultithreadedprogram'sbehaviortobepredictable,accesstosharedvariablesmustbeproperlycoordinatedso thatthreadsdonotinterferewithoneanother.Fortunately,Javaprovidessynchronizationmechanismstocoordinate suchaccess.
UnsafeSequencecanbefixedbymakinggetNextasynchronizedmethod,asshowninSequenceinListing1.2,[3]thus preventingtheunfortunateinteractioninFigure1.1.(ExactlywhythisworksisthesubjectofChapters2and3.)
[3]@GuardedByisdescribedinSection2.4;itdocumentsthesynchronizationpolicyforSequence.
Listing1.2.ThreadǦsafeSequenceGenerator.
@ThreadSafe
public class Sequence {
@GuardedBy("this") private int nextValue;
public synchronized int getNext() {
return nextValue++;
}
}
Intheabsenceofsynchronization,thecompiler,hardware,andruntimeareallowedtotakesubstantiallibertieswith the timing and ordering of actions, such as caching variables in registers or processorͲlocal caches where they are temporarily (or even permanently) invisible to other threads. These tricks are in aid of better performance and are generally desirable, but they place a burden on the developer to clearly identify where data is being shared across threads so that these optimizations do not undermine safety. (Chapter 16 gives the gory details on exactly what ordering guarantees the JVM makes and how synchronization affects those guarantees, but if you follow the rules in Chapters2and3,youcansafelyavoidtheselowͲleveldetails.)
1.3.2.LivenessHazards
It is critically important to pay attention to thread safety issues when developing concurrent code: safety cannot be compromised.TheimportanceofsafetyisnotuniquetomultithreadedprogramsͲsingleͲthreadedprogramsalsomust takecaretopreservesafetyandcorrectnessͲbuttheuseofthreadsintroducesadditionalsafetyhazardsnotpresentin singleͲthreadedprograms.Similarly,theuseofthreadsintroducesadditionalformsoflivenessfailurethatdonotoccur insingleͲthreadedprograms.
While safety means "nothing bad ever happens", liveness concerns the complementary goal that "something good eventuallyhappens".Alivenessfailureoccurswhenanactivitygetsintoastatesuchthatitispermanentlyunableto make forward progress. One form of liveness failure that can occur in sequential programs is an inadvertent infinite loop,wherethecodethatfollowstheloopnevergetsexecuted.Theuseofthreadsintroducesadditionallivenessrisks.
For example, if thread A is waiting for a resource that thread B holds exclusively, and B never releases it, A will wait forever. Chapter 10 describes various forms of liveness failures and how to avoid them, including deadlock (Section 10.1), starvation (Section 10.3.1), and livelock (Section 10.3.3). Like most concurrency bugs, bugs that cause liveness failurescanbeelusivebecausetheydependontherelativetimingofeventsindifferentthreads,andthereforedonot alwaysmanifestthemselvesindevelopmentortesting.
1.3.3.PerformanceHazards
Relatedtolivenessisperformance.Whilelivenessmeansthatsomethinggoodeventuallyhappens,eventuallymaynot be good enough Ͳ we often want good things to happen quickly. Performance issues subsume a broad range of problems, including poor service time, responsiveness, throughput, resource consumption, or scalability. Just as with safety and liveness, multithreaded programs are subject to all the performance hazards of singleͲthreaded programs, andtoothersaswellthatareintroducedbytheuseofthreads.
3BChapter1Ͳ IntroductionͲ12B1.3.RisksofThreads
7
Inwelldesignedconcurrentapplicationstheuseofthreadsisanetperformancegain,butthreadsneverthelesscarry somedegreeofruntime overhead.ContextswitchesͲwhentheschedulersuspendstheactive thread temporarilyso anotherthreadcanrunͲaremorefrequentinapplicationswithmanythreads,andhavesignificantcosts:savingand restoring execution context, loss of locality, and CPU time spent scheduling threads instead of running them. When threads share data, they must use synchronization mechanisms that can inhibit compiler optimizations, flush or invalidate memory caches, and create synchronization traffic on the shared memory bus. All these factors introduce additionalperformancecosts;Chapter11coverstechniquesforanalyzingandreducingthesecosts.
8 JavaConcurrencyInPractice
1.4.ThreadsareEverywhere
Evenifyourprogramneverexplicitlycreatesathread,frameworksmaycreatethreadsonyourbehalf,andcodecalled fromthesethreadsmustbethreadͲsafe.Thiscanplaceasignificantdesignandimplementationburdenondevelopers, sincedevelopingthreadͲsafeclassesrequiresmorecareandanalysisthandevelopingnonͲthreadͲsafeclasses.
Every Java application uses threads. When the JVM starts, it creates threads for JVM housekeeping tasks (garbage collection,finalization)andamainthreadforrunningthemainmethod.TheAWT(AbstractWindowToolkit)andSwing user interface frameworks create threads for managing user interface events. Timer creates threads for executing deferred tasks. Component frameworks, such as servlets and RMI create pools of threads and invoke component methodsinthesethreads.
IfyouusethesefacilitiesͲasmanydevelopersdoͲyouhavetobefamiliarwithconcurrencyandthreadsafety,because theseframeworkscreatethreadsandcallyourcomponentsfromthem.Itwouldbenicetobelievethatconcurrencyis an"optional"or"advanced"languagefeature,buttherealityisthatnearlyallJavaapplicationsaremultithreadedand theseframeworksdonotinsulateyoufromtheneedtoproperlycoordinateaccesstoapplicationstate.
Whenconcurrencyisintroducedintoanapplicationbyaframework,itisusuallyimpossibletorestricttheconcurrencyͲ
awarenesstotheframeworkcode,becauseframeworksbytheirnaturemakecallbackstoapplicationcomponentsthat inturnaccessapplicationstate.Similarly,theneedforthreadsafetydoesnotendwiththecomponentscalledbythe frameworkͲitextendstoallcodepathsthataccesstheprogramstateaccessedbythosecomponents.Thus,theneed forthreadsafetyiscontagious.
Frameworks introduce concurrency into applications by calling application components from framework threads.
Componentsinvariablyaccessapplicationstate,thusrequiringthatallcodepathsaccessingthatstatebethreadͲsafe.
The facilities described below all cause application code to be called from threads not managed by the application.
While the need for thread safety may start with these facilities, it rarely ends there; instead, it ripples through the application.
Timer.Timerisaconvenience mechanismforschedulingtaskstorunatalatertime,eitheronceorperiodically.The introductionofaTimercancomplicateanotherwisesequentialprogram,becauseTimerTasksareexecutedinathread managed by the Timer, not the application. If a TimerTask accesses data that is also accessed by other application threads,thennotonlymusttheTimerTaskdosoinathreadͲsafemanner,butsomustanyotherclassesthataccess that data. Often the easiest way to achieve this is to ensure that objects accessed by the TimerTask are themselves threadͲsafe,thusencapsulatingthethreadsafetywithinthesharedobjects.
ServletsandJavaServerPages(JSPs).Theservletsframeworkisdesignedtohandlealltheinfrastructureofdeployinga web application and dispatching requests from remote HTTP clients. A request arriving at the server is dispatched, perhapsthroughachainoffilters,totheappropriateservletorJSP.Eachservletrepresentsacomponentofapplication logic,andinhighͲvolumewebsites,multipleclientsmayrequiretheservicesofthesameservletatonce.Theservlets specification requires that a servlet be prepared to be called simultaneously from multiple threads. In other words, servletsneedtobethreadͲsafe.
Even if you could guarantee that a servlet was only called from one thread at a time, you would still have to pay attentiontothreadsafetywhenbuildingawebapplication.Servletsoftenaccessstateinformationsharedwithother servlets, such as applicationͲscoped objects (those stored in the ServletContext) or sessionͲscoped objects (those stored in the perͲclient HttpSession). When a servlet accesses objects shared across servlets or requests, it must coordinate access to these objects properly, since multiple requests could be accessing them simultaneously from separate threads. Servlets and JSPs, as well as servlet filters and objects stored in scoped containers like ServletContextandHttpSession,simplyhavetobethreadͲsafe.
RemoteMethodInvocation.RMIletsyouinvokemethodsonobjectsrunninginanotherJVM.Whenyoucallaremote methodwithRMI,themethodargumentsarepackaged(marshaled)intoabytestreamandshippedoverthenetworkto theremoteJVM,wheretheyareunpacked(unmarshaled)andpassedtotheremotemethod.
WhentheRMIcodecallsyourremoteobject,inwhatthreaddoesthatcallhappen?Youdon'tknow,butit'sdefinitely notinathreadyoucreatedͲyourobjectgetscalledinathreadmanagedbyRMI.HowmanythreadsdoesRMIcreate?
CouldthesameremotemethodonthesameremoteobjectbecalledsimultaneouslyinmultipleRMIthreads?[4]
3BChapter1Ͳ Introduction Ͳ 13B1.4.ThreadsareEverywhere 9
[4]Answer:yes,butit'snotallthatclearfromtheJavadocͲyouhavetoreadtheRMIspec.
A remote object must guard against two thread safety hazards: properly coordinating access to state that may be shared with other objects, and properly coordinating access to the state of the remote object itself (since the same object may be called in multiple threads simultaneously). Like servlets, RMI objects should be prepared for multiple simultaneouscallsandmustprovidetheirownthreadsafety.
SwingandAWT.GUIapplicationsareinherentlyasynchronous.Usersmayselectamenuitemorpressabuttonatany time, and they expect that the application will respond promptly even if it is in the middle of doing something else.
SwingandAWTaddressthisproblembycreatingaseparatethreadforhandlinguserͲinitiatedeventsandupdatingthe graphicalviewpresentedtotheuser.
Swing components, such as JTable, are not threadͲsafe. Instead, Swing programs achieve their thread safety by confiningallaccesstoGUIcomponentstotheeventthread.IfanapplicationwantstomanipulatetheGUIfromoutside theeventthread,itmustcausethecodethatwillmanipulatetheGUItorunintheeventthreadinstead.
WhentheuserperformsaUIaction,aneventhandleriscalledintheeventthreadtoperformwhateveroperationthe user requested. If the handler needs to access application state that is also accessed from other threads (such as a documentbeingedited),thentheeventhandler,alongwithanyothercodethataccessesthatstate,mustdosoina threadͲsafemanner.
10 JavaConcurrencyInPractice
PartI:Fundamentals
Chapter2. ThreadSafety
Chapter3. SharingObjects
Chapter4. ComposingObjects
Chapter5. BuildingBlocks
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
11
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
Chapter2.ThreadSafety
Perhapssurprisingly,concurrentprogrammingisn'tsomuchaboutthreadsorlocks,anymorethancivilengineeringis aboutrivetsandIͲbeams.Ofcourse,buildingbridgesthatdon'tfalldownrequiresthecorrectuseofalotofrivetsandIͲ
beams,justasbuildingconcurrentprogramsrequirethecorrectuseofthreadsandlocks.Butthesearejustmechanisms meanstoanend.WritingthreadͲsafecodeis,atitscore,aboutmanagingaccesstostate,andinparticulartoshared, mutablestate.
Informally,anobject'sstateisitsdata,storedinstatevariablessuchasinstanceorstaticfields.Anobject'sstatemay includefieldsfromother,dependentobjects;aHashMap'sstateispartiallystoredintheHashMapobjectitself,butalsoin manyMap.Entryobjects.Anobject'sstateencompassesanydatathatcanaffectitsexternallyvisiblebehavior.
Byshared,wemeanthatavariablecouldbeaccessedbymultiplethreads;bymutable,wemeanthatitsvaluecould changeduringitslifetime.Wemaytalkaboutthreadsafetyasifitwereaboutcode,butwhatwearereallytryingtodo isprotectdatafromuncontrolledconcurrentaccess.
Whether an object needs to be threadͲsafe depends on whether it will be accessed from multiple threads. This is a property of how the object is used in a program, not what it does. Making an object threadͲsafe requires using synchronization to coordinate access to its mutable state; failing to do so could result in data corruption and other undesirableconsequences.
Whenever more than one thread accesses a given state variable, and one of them might write to it, they all must coordinate their access to it using synchronization. The primary mechanism for synchronization in Java is the synchronized keyword, which provides exclusive locking, but the term "synchronization" also includes the use of volatilevariables,explicitlocks,andatomicvariables.
Youshouldavoidthetemptationtothinkthatthereare"special"situationsinwhichthisruledoesnotapply.Aprogram thatomitsneededsynchronizationmightappeartowork,passingitstestsandperformingwellforyears,butitisstill brokenandmayfailatanymoment.
If multiple threads access the same mutable state variable without appropriate synchronization, your program is broken.Therearethreewaystofixit:
x Don'tsharethestatevariableacrossthreads;
x Makethestatevariableimmutable;or
x Usesynchronizationwheneveraccessingthestatevariable.
If you haven't considered concurrent access in your class design, some of these approaches can require significant designmodifications,sofixingtheproblemmightnotbeastrivialasthisadvicemakesitsound.Itisfareasiertodesign aclasstobethreadͲsafethantoretrofititforthreadsafetylater.
Inalargeprogram,identifyingwhethermultiplethreadsmightaccessagivenvariablecanbecomplicated.Fortunately, thesameobjectͲorientedtechniquesthathelpyouwritewellͲorganized,maintainableclassesͲsuchasencapsulation anddatahidingͲcanalsohelpyoucreatethreadͲsafeclasses.Thelesscodethathasaccesstoaparticularvariable,the easieritistoensurethatallofitusesthepropersynchronization,andtheeasieritistoreasonabouttheconditions under which a given variable might be accessed. The Java language doesn't force you to encapsulate state Ͳ it is perfectly allowable to store state in public fields (even public static fields) or publish a reference to an otherwise internalobjectͲbutthebetterencapsulatedyourprogramstate,theeasieritistomakeyourprogramthreadͲsafeand tohelpmaintainerskeepitthatway.
When designing threadͲsafe classes, good objectͲoriented techniques Ͳ encapsulation, immutability, and clear specificationofinvariantsͲareyourbestfriends.
TherewillbetimeswhengoodobjectͲorienteddesigntechniquesareatoddswithrealͲworldrequirements;itmaybe necessary in these cases to compromise the rules of good design for the sake of performance or for the sake of backward compatibility with legacy code. Sometimes abstraction and encapsulation are at odds with performance Ͳ
12 JavaConcurrencyInPractice
althoughnotnearlyasoftenasmanydevelopersbelieveͲbutitisalwaysagoodpracticefirsttomakeyourcoderight, andthen makeitfast.Eventhen,pursueoptimizationonlyifyourperformancemeasurementsandrequirementstell youthatyoumust,andifthosesamemeasurementstellyouthatyouroptimizationsactuallymadeadifferenceunder realisticconditions.[1]
[1] In concurrent code, this practice should be adhered to even more than usual. Because concurrency bugs are so difficult to reproduce and debug,thebenefitofasmallperformancegainonsomeinfrequentlyusedcodepathmaywellbedwarfedbytheriskthattheprogramwillfailin thefield.
Ifyoudecidethatyousimplymustbreakencapsulation,allisnotlost.ItisstillpossibletomakeyourprogramthreadͲ
safe, it is just a lot harder. Moreover, the thread safety of your program will be more fragile, increasing not only developmentcostandriskbutmaintenancecostandriskaswell.Chapter4characterizestheconditionsunderwhichit issafetorelaxencapsulationofstatevariables.
We'veusedtheterms"threadͲsafeclass"and"threadͲsafeprogram"nearlyinterchangeablythusfar.IsathreadͲsafe program one that is constructed entirely of threadͲsafe classes? Not necessarily Ͳ a program that consists entirely of threadͲsafe classes may not be threadͲsafe, and a threadͲsafe program may contain classes that are not threadͲsafe.
TheissuessurroundingthecompositionofthreadͲsafeclassesarealsotakenupinChapter4.Inanycase,theconceptof athreadͲsafeclassmakessenseonlyiftheclassencapsulatesitsownstate.Threadsafetymaybeatermthatisapplied tocode,butitisaboutstate,anditcanonlybeappliedtotheentirebodyofcodethatencapsulatesitsstate,whichmay beanobjectoranentireprogram.
2.1.WhatisThreadSafety?
Defining thread safety is surprisingly tricky. The more formal attempts are so complicated as to offer little practical guidance or intuitive understanding, and the rest are informal descriptions that can seem downright circular. A quick Googlesearchturnsupnumerous"definitions"likethese:
...canbecalledfrommultipleprogramthreadswithoutunwantedinteractionsbetweenthethreads.
...maybecalledbymorethanonethreadatatimewithoutrequiringanyotheractiononthecaller'spart.
Given definitions like these, it's no wonder we find thread safety confusing! They sound suspiciously like "a class is threadͲsafeifitcanbeusedsafelyfrommultiplethreads."Youcan'treallyarguewithsuchastatement,butitdoesn't offermuchpracticalhelp either.How dowetella threadͲsafe classfroman unsafeone? Whatdowe evenmeanby
"safe"?
Attheheartofanyreasonabledefinitionofthreadsafetyistheconceptofcorrectness.Ifourdefinitionofthreadsafety isfuzzy,itisbecausewelackacleardefinitionofcorrectness.
Correctness means that a class conforms to its specification. A good specification defines invariants constraining an object's state and post Ͳ conditions describing the effects of its operations. Since we often don't write adequate specificationsforourclasses,howcanwepossiblyknowtheyarecorrect?Wecan't,butthatdoesn'tstopusfromusing themanywayoncewe'veconvincedourselvesthat"thecodeworks".This"codeconfidence"isaboutascloseasmany ofusgettocorrectness,solet'sjustassumethatsingleͲthreadedcorrectnessissomethingthat"weknowitwhenwe see it". Having optimistically defined "correctness" as something that can be recognized, we can now define thread safetyinasomewhatlesscircularway:aclassisthreadͲsafewhenitcontinuestobehavecorrectlywhenaccessedfrom multiplethreads.
A class is threadͲsafe if it behaves correctly when accessed from multiple threads, regardless of the scheduling or interleavingoftheexecutionofthosethreadsbytheruntimeenvironment,andwithnoadditionalsynchronizationor othercoordinationonthepartofthecallingcode.
Since any singleͲthreaded program is also a valid multithreaded program, it cannot be threadͲsafe if it is not even correctinasingleͲthreadedenvironment.[2]Ifanobjectiscorrectlyimplemented,nosequenceofoperationsͲcallsto publicmethodsandreadsorwritesofpublicfieldsͲshouldbeabletoviolateanyofitsinvariantsorpostͲconditions.No setofoperationsperformedsequentiallyorconcurrentlyoninstancesofathreadͲsafeclasscancauseaninstancetobe inaninvalidstate.
[2]Ifthelooseuseof"correctness"herebothersyou,youmayprefertothinkofathreadͲsafeclassasonethatisnomorebrokeninaconcurrent environmentthaninasingleͲthreadedenvironment.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
13
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
ThreadͲsafeclassesencapsulateanyneededsynchronizationsothatclientsneednotprovidetheirown.
2.1.1.Example:AStatelessServlet
In Chapter 1, we listed a number of frameworks that create threads and call your components from those threads, leaving you with the responsibility of making your components threadͲsafe. Very often, threadͲsafety requirements stemnotfromadecisiontousethreadsdirectlybutfromadecisiontouseafacilityliketheServletsframework.We're goingtodevelopasimpleexampleͲaservletͲbasedfactorizationserviceͲandslowlyextendittoaddfeatureswhile preservingitsthreadsafety.
Listing2.1showsoursimplefactorizationservlet.Itunpacksthenumbertobefactoredfromtheservletrequest,factors it,andpackagestheresultsintotheservletresponse.
Listing2.1.AStatelessServlet.
@ThreadSafe
public class StatelessFactorizer implements Servlet {
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
BigInteger[] factors = factor(i);
encodeIntoResponse(resp, factors);
}
}
StatelessFactorizeris,likemostservlets,stateless:ithasnofieldsandreferencesnofieldsfromotherclasses.The transientstateforaparticularcomputationexistssolelyinlocalvariablesthatarestoredonthethread'sstackandare accessibleonlytothe executing thread.OnethreadaccessingaStatelessFactorizer cannotinfluencetheresult of anotherthreadaccessingthesameStatelessFactorizer;becausethetwothreadsdonotsharestate,itisasifthey were accessing different instances. Since the actions of a thread accessing a stateless object cannot affect the correctnessofoperationsinotherthreads,statelessobjectsarethreadͲsafe.
StatelessobjectsarealwaysthreadͲsafe.
The fact that most servlets can be implemented with no state greatly reduces the burden of making servlets threadͲ
safe.Itisonlywhenservletswanttorememberthingsfromonerequesttoanotherthatthethreadsafetyrequirement becomesanissue.
2.2.Atomicity
What happens when we add one element of state to what was a stateless object? Suppose we want to add a "hit counter"thatmeasuresthenumberofrequestsprocessed.Theobviousapproachistoaddalongfieldtotheservlet andincrementitoneachrequest,asshowninUnsafeCountingFactorizerinListing2.2.
Unfortunately,UnsafeCountingFactorizerisnotthreadͲsafe,eventhoughitwouldworkjustfineinasingleͲthreaded environment. Just like UnsafeSequence on page 6, it is susceptible to lost updates. While the increment operation,
++count, may look like a single action because of its compact syntax, it is not atomic, which means that it does not executeasasingle,indivisibleoperation.Instead,itisshorthandforasequenceofthreediscreteoperations:fetchthe currentvalue,addonetoit,andwritethenewvalueback.ThisisanexampleofareadͲmodifyͲwriteoperation,inwhich theresultingstateisderivedfromthepreviousstate.
14 JavaConcurrencyInPractice
Listing2.2.ServletthatCountsRequestswithouttheNecessarySynchronization.
@NotThreadSafe
public class UnsafeCountingFactorizer implements Servlet {
private long count = 0;
public long getCount() { return count; }
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
BigInteger[] factors = factor(i);
++count;
encodeIntoResponse(resp, factors);
}
}
Figure 1.1 on page 6 shows what can happen if two threads try to increment a counter simultaneously without synchronization.Ifthecounterisinitially9,withsomeunluckytimingeachthreadcouldreadthevalue,seethatitis9, addonetoit,andeachsetthecounterto10.Thisisclearlynotwhatissupposedtohappen;anincrementgotlostalong theway,andthehitcounterisnowpermanentlyoffbyone.
YoumightthinkthathavingaslightlyinaccuratecountofhitsinawebͲbasedserviceisanacceptablelossofaccuracy, andsometimesitis.Butifthecounterisbeingusedtogeneratesequencesoruniqueobjectidentifiers,returningthe samevaluefrommultipleinvocationscouldcauseseriousdataintegrityproblems.[3]Thepossibilityofincorrectresults inthepresenceofunluckytimingissoimportantinconcurrentprogrammingthatithasaname:aracecondition.
[3] The approach taken by UnsafeSequence and UnsafeCountingFactorizer has other serious problems, including the possibility of stale data (Section3.1.1).
2.2.1.RaceConditions
UnsafeCountingFactorizerhasseveralraceconditionsthatmakeitsresultsunreliable.Araceconditionoccurswhen thecorrectnessofacomputationdependsontherelativetimingorinterleavingofmultiplethreadsbytheruntime;in otherwords,whengettingtherightanswerreliesonluckytiming.[4]ThemostcommontypeofraceconditionischeckͲ
thenͲact,whereapotentiallystaleobservationisusedtomakeadecisiononwhattodonext.
[4] The term race condition is often confused with the related term data race, which arises when synchronization is not used to coordinate all accesstoasharednonͲfinalfield.Youriskadataracewheneverathreadwritesavariablethatmightnextbereadbyanotherthreadorreadsa variable that might have last been written by another thread if both threads do not use synchronization; code with data races has no useful definedsemanticsundertheJavaMemoryModel.Notallraceconditionsaredataraces,andnotalldataracesareraceconditions,buttheyboth cancauseconcurrentprogramstofailinunpredictableways.UnsafeCountingFactorizerhasbothraceconditionsanddataraces.SeeChapter 16formoreondataraces.
We often encounter race conditions in real life. Let's say you planned to meet a friend at noon at the Starbucks on UniversityAvenue.Butwhenyougetthere,yourealizetherearetwoStarbucksonUniversityAvenue,andyou'renot surewhichoneyouagreedtomeetat.At12:10,youdon'tseeyourfriendatStarbucksA,soyouwalkovertoStarbucks B to see if he's there, but he isn't there either. There are a few possibilities: your friend is late and not at either Starbucks;yourfriendarrivedatStarbucksAafteryouleft;oryourfriendwasatStarbucksB,butwenttolookforyou, andisnowenroutetoStarbucksA.Let'sassumetheworstandsayitwasthelastpossibility.Nowit's12:15,you'veboth beentobothStarbucks,andyou'rebothwonderingifyou'vebeenstoodup.Whatdoyoudonow?Gobacktotheother Starbucks?Howmanytimesareyougoingtogobackandforth?Unlessyouhaveagreedonaprotocol,youcouldboth spendthedaywalkingupanddownUniversityAvenue,frustratedandundercaffeinated.
Theproblemwiththe"I'lljustnipupthestreetandseeifhe'sattheotherone"approachisthatwhileyou'rewalkingup thestreet,yourfriendmighthavemoved.YoulookaroundStarbucksA,observe"he'snothere",andgolookingforhim.
AndyoucandothesameforStarbucksB,butnotatthesametime.Ittakesafewminutestowalkupthestreet,and duringthosefewminutes,thestateofthesystemmayhavechanged.
TheStarbucksexampleillustratesaraceconditionbecausereachingthedesiredoutcome(meetingyourfriend)depends ontherelativetimingofevents(wheneachofyouarrivesatoneStarbucksortheother,howlongyouwaittherebefore switching,etc).TheobservationthatheisnotatStarbucksAbecomespotentiallyinvalidassoonasyouwalkoutthe frontdoor;hecouldhavecomeinthroughthebackdoorandyouwouldn'tknow.Itisthisinvalidationofobservations that characterizes most race conditions Ͳ using a potentially stale observation to make a decision or perform a computation.ThistypeofraceconditioniscalledcheckͲthenͲact:youobservesomethingtobetrue(fileXdoesn'texist)
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
15
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
and then take action based on that observation (create X); but in fact the observation could have become invalid betweenthetimeyouobserveditandthetimeyouactedonit(someoneelsecreatedXinthemeantime),causinga problem(unexpectedexception,overwrittendata,filecorruption).
2.2.2.Example:RaceConditionsinLazyInitialization
A common idiom that uses checkͲthenͲact is lazy initialization. The goal of lazy initialization is to defer initializing an objectuntilitisactuallyneededwhileatthesametimeensuringthatitisinitializedonlyonce.LazyInitRaceinListing 2.3 illustrates the lazy initialization idiom. The getInstance method first checks whether the ExpensiveObject has alreadybeeninitialized,inwhichcaseitreturnstheexistinginstance;otherwiseitcreatesanewinstanceandreturnsit afterretainingareferencetoitsothatfutureinvocationscanavoidthemoreexpensivecodepath.
Listing2.3.RaceConditioninLazyInitialization.
@NotThreadSafe
public class LazyInitRace {
private ExpensiveObject instance = null;
public ExpensiveObject getInstance() {
if (instance == null)
instance = new ExpensiveObject();
return instance;
}
}
LazyInitRacehasraceconditionsthatcanundermineitscorrectness.SaythatthreadsAandBexecutegetInstanceat the same time. A sees that instance is null, and instantiates a new ExpensiveObject. B also checks if instance is null.Whetherinstanceisnullatthispointdependsunpredictablyontiming,includingthevagariesofschedulingand howlongAtakestoinstantiatetheExpensiveObjectandsettheinstancefield.IfinstanceisnullwhenBexamines it,thetwocallerstogetInstancemayreceivetwodifferentresults,eventhoughgetInstanceisalwayssupposedto returnthesameinstance.
The hitͲcounting operation in UnsafeCountingFactorizer has another sort of race condition. ReadͲmodifyͲwrite operations,likeincrementingacounter,defineatransformationofanobject'sstateintermsofitspreviousstate.To incrementacounter,youhavetoknowitspreviousvalueandmakesurenooneelsechangesorusesthatvaluewhile youareinmidͲupdate.
Likemostconcurrencyerrors,raceconditionsdon'talwaysresultinfailure:someunluckytimingisalsorequired.But raceconditionscancauseseriousproblems.IfLazyInitRaceisusedtoinstantiateanapplicationͲwideregistry,havingit return different instances from multiple invocations could cause registrations to be lost or multiple activities to have inconsistent views of the set of registered objects. If UnsafeSequence is used to generate entity identifiers in a persistenceframework,twodistinctobjectscouldendupwiththesameID,violatingidentityintegrityconstraints.
2.2.3.CompoundActions
BothLazyInitRaceandUnsafeCountingFactorizercontainedasequenceofoperationsthatneededtobeatomic,or indivisible, relative to other operations on the same state. To avoid race conditions, there must be a way to prevent otherthreadsfromusingavariablewhilewe'reinthemiddleofmodifyingit,sowecanensurethatotherthreadscan observeormodifythestateonlybeforewestartorafterwefinish,butnotinthemiddle.
Operations A and B are atomic with respect to each other if, from the perspective of a thread executing A, when anotherthreadexecutesB,eitherallofBhasexecutedornoneofithas.Anatomicoperationisonethatisatomicwith respecttoalloperations,includingitself,thatoperateonthesamestate.
IftheincrementoperationinUnsafeSequencewereatomic,theraceconditionillustratedinFigure1.1onpage6could notoccur,andeachexecutionoftheincrementoperationwouldhavethedesiredeffectofincrementingthecounterby exactly one. To ensure thread safety, checkͲthenͲact operations (like lazy initialization) and readͲmodifyͲwrite operations (like increment) must always be atomic. We refer collectively to checkͲthenͲact and readͲmodifyͲwrite sequencesascompoundactions:sequencesofoperationsthatmustbeexecutedatomicallyinordertoremainthreadͲ
16 JavaConcurrencyInPractice
safe.Inthenextsection,we'llconsiderlocking,Java'sbuiltͲinmechanismforensuringatomicity.Fornow,we'regoingto fixtheproblemanotherway,byusinganexistingthreadͲsafeclass,asshowninCountingFactorizerinListing2.4.
Listing2.4.ServletthatCountsRequestsUsingAtomicLong.
@ThreadSafe
public class CountingFactorizer implements Servlet {
private final AtomicLong count = new AtomicLong(0);
public long getCount() { return count.get(); }
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
BigInteger[] factors = factor(i);
count.incrementAndGet();
encodeIntoResponse(resp, factors);
}
}
Thejava.util.concurrent.atomicpackagecontainsatomicvariableclassesforeffectingatomicstatetransitionson numbers and object references. By replacing the long counter with an AtomicLong, we ensure that all actions that accessthecounterstateareatomic.[5]Becausethestateoftheservletisthestateofthecounterandthecounteris threadͲsafe,ourservletisonceagainthreadͲsafe.
[5]CountingFactorizercallsincrementAndGettoincrementthecounter,whichalsoreturnstheincrementedvalue;inthiscasethereturnvalueis ignored.
WewereabletoaddacountertoourfactoringservletandmaintainthreadsafetybyusinganexistingthreadͲsafeclass to manage the counter state,AtomicLong. When a single element of state is added to a stateless class, the resulting classwillbethreadͲsafeifthestateisentirelymanagedbyathreadͲsafeobject.But,aswe'llseeinthenextsection, goingfromonestatevariabletomorethanoneisnotnecessarilyassimpleasgoingfromzerotoone.
Wherepractical,useexistingthreadͲsafeobjects,likeAtomicLong,tomanageyourclass'sstate.Itissimplertoreason aboutthepossiblestatesandstatetransitionsforexistingthreadͲsafeobjectsthanitisforarbitrarystatevariables,and thismakesiteasiertomaintainandverifythreadsafety.
2.3.Locking
WewereabletoaddonestatevariabletoourservletwhilemaintainingthreadsafetybyusingathreadͲsafeobjectto managetheentirestateoftheservlet.Butifwewanttoaddmorestatetoourservlet,canwejustaddmorethreadͲ
safestatevariables?
Imaginethatwewanttoimprovetheperformanceofourservletbycachingthemostrecentlycomputedresult,justin case two consecutive clients request factorization of the same number. (This is unlikely to be an effective caching strategy;weofferabetteroneinSection5.6.)Toimplementthisstrategy,weneedtoremembertwothings:thelast numberfactored,anditsfactors.
We used AtomicLong to manage the counter state in a threadͲsafe manner; could we perhaps use its cousin, AtomicReference, [6] to manage the last number and its factors? An attempt at this is shown in UnsafeCachingFactorizerinListing2.5.
[6]JustasAtomicLongisathreadͲsafeholderclassforalonginteger,AtomicReferenceisathreadsafeholderclassforanobjectreference.Atomic variablesandtheirbenefitsarecoveredinChapter15.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
17
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
Listing2.5.ServletthatAttemptstoCacheitsLastResultwithoutAdequateAtomicity.
@NotThreadSafe
public class UnsafeCachingFactorizer implements Servlet {
private final AtomicReference<BigInteger> lastNumber
= new AtomicReference<BigInteger>();
private final AtomicReference<BigInteger[]> lastFactors
= new AtomicReference<BigInteger[]>();
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
if (i.equals(lastNumber.get()))
encodeIntoResponse(resp, lastFactors.get() );
else {
BigInteger[] factors = factor(i);
lastNumber.set(i);
lastFactors.set(factors);
encodeIntoResponse(resp, factors);
}
}
}
Unfortunately, this approach does not work. Even though the atomic references are individually threadͲsafe, UnsafeCachingFactorizerhasraceconditionsthatcouldmakeitproducethewronganswer.
Thedefinitionofthreadsafetyrequiresthatinvariantsbepreservedregardlessoftimingorinterleavingofoperationsin multiplethreads.OneinvariantofUnsafeCachingFactorizeristhattheproductofthefactorscachedinlastFactors equalthevaluecachedinlastNumber;ourservletiscorrectonlyifthisinvariantalwaysholds.Whenmultiplevariables participate in an invariant, they are not independent: the value of one constrains the allowed value(s) of the others.
Thuswhenupdatingone,youmustupdatetheothersinthesameatomicoperation.
With some unlucky timing,UnsafeCachingFactorizer can violate this invariant. Using atomic references, we cannot update both lastNumber and lastFactors simultaneously, even though each call to set is atomic; there is still a windowofvulnerabilitywhenonehasbeenmodifiedandtheotherhasnot,andduringthattimeotherthreadscould see that the invariant does not hold. Similarly, the two values cannot be fetched simultaneously: between the time whenthreadAfetchesthetwovalues,threadBcouldhavechangedthem,andagainAmayobservethattheinvariant doesnothold.
Topreservestateconsistency,updaterelatedstatevariablesinasingleatomicoperation.
2.3.1.IntrinsicLocks
Java provides a builtͲin locking mechanism for enforcing atomicity: the synchronized block. (There is also another critical aspect to locking and other synchronization mechanismsvisibility Ͳ which is covered in Chapter 3.) A synchronized block has two parts: a reference to an object that will serve as the lock, and a block of code to be guarded by that lock. A synchronized method is shorthand for a synchronized block that spans an entire method body,andwhoselockistheobjectonwhichthemethodisbeinginvoked.(StaticsynchronizedmethodsusetheClass objectforthelock.)
synchronized (lock) {
// Access or modify shared state guarded by lock
}
EveryJavaobjectcanimplicitlyactasalockforpurposesofsynchronization;thesebuiltͲinlocksarecalledintrinsiclocks ormonitorlocks.Thelockisautomaticallyacquiredbytheexecutingthreadbeforeenteringasynchronizedblockand automaticallyreleasedwhencontrolexitsthesynchronizedblock,whetherbythenormalcontrolpathorbythrowing an exception out of the block. The only way to acquire an intrinsic lock is to enter a synchronized block or method guardedbythatlock.
IntrinsiclocksinJavaactasmutexes(ormutualexclusionlocks),whichmeansthatatmostonethreadmayownthe lock.WhenthreadAattemptstoacquirealockheldbythreadB,Amustwait,orblock,untilBreleasesit.IfBnever releasesthelock,Awaitsforever.
18 JavaConcurrencyInPractice
Sinceonlyonethreadatatimecanexecuteablockofcodeguardedbyagivenlock,thesynchronizedblocksguarded bythesamelockexecuteatomicallywithrespecttooneanother.Inthecontextofconcurrency,atomicitymeansthe samethingasitdoesintransactionalapplicationsͲthatagroupofstatementsappeartoexecuteasasingle,indivisible unit. No thread executing a synchronized block can observe another thread to be in the middle of a synchronized blockguardedbythesamelock.
Themachineryofsynchronizationmakesiteasytorestorethreadsafetytothefactoringservlet.Listing2.6makesthe service method synchronized, so only one thread may enter service at a time. SynchronizedFactorizer is now threadͲsafe;however,thisapproachisfairlyextreme,sinceitinhibitsmultipleclientsfromusingthefactoringservlet simultaneouslyatallͲresultinginunacceptablypoorresponsiveness.ThisproblemͲwhichisaperformanceproblem, notathreadsafetyproblemͲisaddressedinSection2.5.
Listing2.6.ServletthatCachesLastResult,ButwithUnacceptablyPoorConcurrency.
@ThreadSafe
public class SynchronizedFactorizer implements Servlet {
@GuardedBy("this") private BigInteger lastNumber;
@GuardedBy("this") private BigInteger[] lastFactors;
public synchronized void service(ServletRequest req,
ServletResponse resp) {
BigInteger i = extractFromRequest(req);
if (i.equals(lastNumber))
encodeIntoResponse(resp, lastFactors);
else {
BigInteger[] factors = factor(i);
lastNumber = i;
lastFactors = factors;
encodeIntoResponse(resp, factors);
}
}
}
2.3.2.Reentrancy
When a thread requests a lock that is already held by another thread, the requesting thread blocks. But because intrinsiclocksarereentrant,ifathreadtriestoacquirealockthatitalreadyholds,therequestsucceeds.Reentrancy means that locks are acquired on a perͲthread rather than perͲinvocation basis. [7] Reentrancy is implemented by associatingwitheachlockanacquisitioncountandanowningthread.Whenthecountiszero,thelockisconsidered unheld.Whenathreadacquiresapreviouslyunheldlock,theJVMrecordstheownerandsetstheacquisitioncountto one. If that same thread acquires the lock again, the count is incremented, and when the owning thread exits the synchronizedblock,thecountisdecremented.Whenthecountreacheszero,thelockisreleased.
[7]Thisdiffersfromthedefaultlockingbehaviorforpthreads(POSIXthreads)mutexes,whicharegrantedonaperͲinvocationbasis.
Reentrancy facilitates encapsulation of locking behavior, and thus simplifies the development of objectͲoriented concurrentcode.Withoutreentrantlocks,theverynaturalͲlookingcodeinListing2.7,inwhichasubclassoverridesa synchronized method and then calls the superclass method, would deadlock. Because the doSomething methods in Widget and LoggingWidget are both synchronized, each tries to acquire the lock on the Widget before proceeding.
Butifintrinsiclockswerenotreentrant,thecalltosuper.doSomethingwouldneverbeabletoacquirethelockbecause it would be considered already held, and the thread would permanently stall waiting for a lock it can never acquire.
Reentrancysavesusfromdeadlockinsituationslikethis.
Listing2.7.CodethatwouldDeadlockifIntrinsicLockswereNotReentrant.
public class Widget {
public synchronized void doSomething() {
...
}
}
public class LoggingWidget extends Widget {
public synchronized void doSomething() {
System.out.println(toString() + ": calling doSomething"); super.doSomething();
}
}
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
19
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
2.4.GuardingStatewithLocks
Because locks enable serialized [8] access to the code paths they guard, we can use them to construct protocols for guaranteeingexclusiveaccesstosharedstate.Followingtheseprotocolsconsistentlycanensurestateconsistency.
[8]Serializingaccesstoanobjecthasnothingtodowithobjectserialization(turninganobjectintoabytestream);serializingaccessmeansthat threadstaketurnsaccessingtheobjectexclusively,ratherthandoingsoconcurrently.
Compoundactionsonsharedstate,suchasincrementingahitcounter(readͲmodifyͲwrite)orlazyinitialization(checkͲ
thenͲact),mustbemadeatomictoavoidraceconditions.Holdingalockfortheentiredurationofacompoundaction canmakethatcompoundactionatomic.However,justwrappingthecompoundactionwithasynchronizedblockisnot sufficient;ifsynchronizationisusedtocoordinateaccesstoavariable,itisneededeverywherethatvariableisaccessed.
Further, when using locks to coordinate access to a variable, the same lock must be used wherever that variable is accessed.
Itisacommonmistaketoassumethatsynchronizationneedstobeusedonlywhenwritingtosharedvariables;thisis simplynottrue.(ThereasonsforthiswillbecomeclearerinSection3.1.)
Foreachmutablestatevariablethatmaybeaccessedbymorethanonethread,allaccessestothatvariablemustbe performedwiththesamelockheld.Inthiscase,wesaythatthevariableisguardedbythatlock.
InSynchronizedFactorizerinListing2.6,lastNumberandlastFactorsareguardedbytheservletobject'sintrinsic lock;thisisdocumentedbythe@GuardedByannotation.
Thereisnoinherentrelationshipbetweenanobject'sintrinsiclockanditsstate;anobject'sfieldsneednotbeguarded byitsintrinsiclock,thoughthisisaperfectlyvalidlockingconventionthatisusedbymanyclasses.Acquiringthelock associatedwithanobjectdoesnotpreventotherthreadsfromaccessingthatobjectͲtheonlythingthatacquiringa lockpreventsanyotherthreadfromdoingisacquiringthatsamelock.ThefactthateveryobjecthasabuiltͲinlockis justaconveniencesothatyouneedn'texplicitlycreatelockobjects.[9]Itisuptoyoutoconstructlockingprotocolsor synchronizationpoliciesthatletyouaccesssharedstatesafely,andtousethemconsistentlythroughoutyourprogram.
[9] In retrospect, this design decision was probably a bad one: not only can it be confusing, but it forces JVM implementers to make tradeoffs betweenobjectsizeandlockingperformance.
Everyshared,mutablevariableshouldbeguardedbyexactlyonelock.Makeitcleartomaintainerswhichlockthatis.
A common locking convention is to encapsulate all mutable state within an object and to protect it from concurrent accessbysynchronizinganycodepaththataccessesmutablestateusingtheobject'sintrinsiclock.Thispatternisused bymanythreadͲsafeclasses,suchasVectorandothersynchronizedcollectionclasses.Insuchcases,allthevariablesin an object's state are guarded by the object's intrinsic lock. However, there is nothing special about this pattern, and neitherthecompilernortheruntimeenforcesthis (oranyother)patternoflocking.[10]It isalsoeasytosubvertthis lockingprotocolaccidentallybyaddinganewmethodorcodepathandforgettingtousesynchronization.
[10]CodeauditingtoolslikeFindBugscanidentifywhenavariableisfrequentlybutnotalwaysaccessedwithalockheld,whichmayindicatea bug.
NotalldataneedstobeguardedbylocksͲonlymutabledatathatwillbeaccessedfrommultiplethreads.InChapter1, wedescribedhowaddingasimpleasynchronouseventsuchasaTimerTaskcancreatethreadsafetyrequirementsthat ripple throughout your program, especially if your program state is poorly encapsulated. Consider a singleͲthreaded programthatprocessesalargeamountofdata.SingleͲthreadedprogramsrequirenosynchronization,becausenodata issharedacrossthreads.Nowimagineyouwanttoaddafeaturetocreateperiodicsnapshotsofitsprogress,sothatit doesnothavetostartagainfromthebeginningifitcrashesormustbestopped.Youmightchoosetodothiswitha TimerTaskthatgoesoffeverytenminutes,savingtheprogramstatetoafile.
SincetheTimerTaskwillbecalledfromanotherthread(onemanagedbyTimer),anydatainvolvedinthesnapshotis now accessed by two threads: the main program thread and the Timer thread. This means that not only must the TimerTaskcodeusesynchronization whenaccessingthe programstate, butsomustanycode pathintherestof the program that touches that same data. What used to require no synchronization now requires synchronization throughouttheprogram.
20 JavaConcurrencyInPractice
When a variable is guarded by a lock Ͳ meaning that every access to that variable is performed with that lock held Ͳ
you'veensuredthatonlyonethreadatatimecanaccessthatvariable.Whenaclasshasinvariantsthatinvolvemore thanonestatevariable,thereisanadditionalrequirement:eachvariableparticipatingintheinvariantmustbeguarded by the same lock. This allows you to access or update them in a single atomic operation, preserving the invariant.
SynchronizedFactorizerdemonstratesthisrule:boththecachednumberandthecachedfactorsareguardedbythe servletobject'sintrinsiclock.
Foreveryinvariantthatinvolvesmorethanonevariable,allthevariablesinvolvedinthatinvariantmustbeguardedby thesamelock.
Ifsynchronizationisthecureforraceconditions,whynotjustdeclareeverymethodsynchronized?Itturnsoutthat such indiscriminate application of synchronized might be either too much or too little synchronization. Merely synchronizingeverymethod,asVectordoes,isnotenoughtorendercompoundactionsonaVectoratomic: if (!vector.contains(element))
vector.add(element);
ThisattemptataputͲifͲabsentoperationhasaracecondition,eventhoughbothcontainsandaddareatomic.While synchronizedmethodscanmakeindividualoperationsatomic,additionallockingisrequiredͲwhenmultipleoperations are combined into a compound action. (See Section 4.4 for some techniques for safely adding additional atomic operationstothreadͲsafeobjects.)Atthesametime,synchronizingeverymethodcanleadtolivenessorperformance problems,aswesawinSynchronizedFactorizer.
2.5.LivenessandPerformance
In UnsafeCachingFactorizer, we introduced some caching into our factoring servlet in the hope of improving performance.Cachingrequiredsomesharedstate,whichinturnrequiredsynchronizationtomaintaintheintegrityof that state. But the way we used synchronization in SynchronizedFactorizer makes it perform badly. The synchronization policy for SynchronizedFactorizer is to guard each state variable with the servlet object's intrinsic lock, and that policy was implemented by synchronizing the entirety of the service method. This simple, coarseͲ
grainedapproachrestoredsafety,butatahighprice.
Becauseserviceissynchronized,onlyonethreadmayexecuteitatonce.Thissubvertstheintendeduseoftheservlet frameworkͲthatservletsbeabletohandlemultiplerequestssimultaneouslyͲandcanresultinfrustratedusersifthe loadishighenough.Iftheservletisbusyfactoringalargenumber,otherclientshavetowaituntilthecurrentrequestis completebeforetheservletcanstartonthenewnumber.IfthesystemhasmultipleCPUs,processorsmayremainidle eveniftheloadishigh.Inanycase,evenshortͲrunningrequests,suchasthoseforwhichthevalueiscached,maytake anunexpectedlylongtimebecausetheymustwaitforpreviouslongͲrunningrequeststocomplete.
Figure2.1showswhathappenswhenmultiplerequestsarriveforthesynchronizedfactoringservlet:theyqueueupand are handled sequentially. We would describe this web application as exhibiting poor concurrency: the number of simultaneous invocations is limited not by the availability of processing resources, but by the structure of the applicationitself.Fortunately,it iseasytoimprovethe concurrencyof theservletwhile maintainingthreadsafety by narrowing the scope of the synchronized block. You should be careful not to make the scope of the synchronized blocktoosmall;youwouldnotwanttodivideanoperationthatshouldbeatomicintomorethanonesynchronized block.ButitisreasonabletotrytoexcludefromsynchronizedblockslongͲrunningoperationsthatdonotaffectshared state,sothatotherthreadsarenotpreventedfromaccessingthesharedstatewhilethelongͲrunningoperationisin progress.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
21
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety
Figure2.1.PoorConcurrencyofSynchronizedFactorizer.
CachedFactorizer in Listing 2.8 restructures the servlet to use two separate synchronized blocks, each limited to a shortsectionofcode.OneguardsthecheckͲthenͲactsequencethattestswhetherwecanjustreturnthecachedresult, andtheotherguardsupdatingboththecachednumberandthecachedfactors.Asabonus,we'vereintroducedthehit counterandaddeda"cachehit"counteraswell,updatingthemwithintheinitialsynchronizedblock.Becausethese counters constitute shared mutable state as well, we must use synchronization everywhere they are accessed. The portionsofcodethatareoutsidethesynchronizedblocksoperateexclusivelyonlocal(stackͲbased)variables,which arenotsharedacrossthreadsandthereforedonotrequiresynchronization.
Listing2.8.ServletthatCachesitsLastRequestandResult.
@ThreadSafe
public class CachedFactorizer implements Servlet {
@GuardedBy("this") private BigInteger lastNumber;
@GuardedBy("this") private BigInteger[] lastFactors;
@GuardedBy("this") private long hits;
@GuardedBy("this") private long cacheHits;
public synchronized long getHits() { return hits; }
public synchronized double getCacheHitRatio() {
return (double) cacheHits / (double) hits;
}
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
BigInteger[] factors = null;
synchronized (this) {
++hits;
if (i.equals(lastNumber)) {
++cacheHits;
factors = lastFactors.clone();
}
}
if (factors == null) {
factors = factor(i);
synchronized (this) {
lastNumber = i;
lastFactors = factors.clone();
}
}
encodeIntoResponse(resp, factors);
}
}
CachedFactorizernolongerusesAtomicLongforthehitcounter,insteadrevertingtousingalongfield.Itwouldbe safe to use AtomicLong here, but there is less benefit than there was in CountingFactorizer. Atomic variables are useful for effecting atomic operations on a single variable, but since we are already using synchronized blocks to constructatomicoperations,usingtwodifferentsynchronizationmechanismswouldbeconfusingandwouldofferno performanceorsafetybenefit.
TherestructuringofCachedFactorizerprovidesabalancebetweensimplicity(synchronizingtheentiremethod)and concurrency(synchronizingtheshortestpossiblecodepaths).Acquiringandreleasingalockhassomeoverhead,soitis undesirable tobreakdownsynchronizedblockstoofar(suchasfactoring++hitsintoitsownsynchronizedblock), evenifthiswouldnotcompromiseatomicity.CachedFactorizerholdsthelockwhenaccessingstatevariablesandfor thedurationofcompoundactions,butreleasesitbeforeexecutingthepotentiallylongͲrunningfactorizationoperation.
22 JavaConcurrencyInPractice
Thispreservesthreadsafetywithoutundulyaffectingconcurrency;thecodepathsineachofthesynchronizedblocks are"shortenough".
Deciding how big or small to make synchronized blocks may require tradeoffs among competing design forces, includingsafety(whichmustnotbecompromised),simplicity,andperformance.Sometimessimplicityandperformance areatoddswitheachother,althoughasCachedFactorizerillustrates,areasonablebalancecanusuallybefound.
Thereisfrequentlyatensionbetweensimplicityandperformance.Whenimplementingasynchronizationpolicy,resist thetemptationtoprematurelysacrificesimplicity(potentiallycompromisingsafety)forthesakeofperformance.
Wheneveryouuselocking,youshouldbeawareofwhatthecodeintheblockisdoingandhowlikelyitistotakealong timetoexecute.Holdingalockforalongtime,eitherbecauseyouaredoingsomethingcomputeͲintensiveorbecause youexecuteapotentiallyblockingoperation,introducestheriskoflivenessorperformanceproblems.
Avoid holding locks during lengthy computations or operations at risk of not completing quickly such as network or consoleI/O.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
23
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
Chapter3.SharingObjects
WestatedatthebeginningofChapter2thatwritingcorrectconcurrentprogramsisprimarilyaboutmanagingaccessto shared, mutable state. That chapter was about using synchronization to prevent multiple threads from accessing the samedataatthesametime;thischapterexaminestechniquesforsharingandpublishingobjectssotheycanbesafely accessedbymultiplethreads.Together,theylaythefoundationforbuildingthreadͲsafeclassesandsafelystructuring concurrentapplicationsusingthejava.util.concurrentlibraryclasses.
We have seen how synchronized blocks and methods can ensure that operations execute atomically, but it is a commonmisconceptionthatsynchronizedisonlyaboutatomicityordemarcating"criticalsections".Synchronization also has another significant, and subtle, aspect: memory visibility. We want not only to prevent one thread from modifyingthestateofanobjectwhenanotherisusingit,butalsotoensurethatwhenathreadmodifiesthestateofan object,otherthreadscanactuallyseethechangesthatweremade.Butwithoutsynchronization,thismaynothappen.
Youcanensurethatobjectsarepublishedsafelyeitherbyusingexplicitsynchronizationorbytakingadvantageofthe synchronizationbuiltintolibraryclasses.
3.1.Visibility
Visibilityissubtlebecausethethingsthatcangowrongaresocounterintuitive.InasingleͲthreadedenvironment,ifyou writeavaluetoavariableandlaterreadthatvariablewithnointerveningwrites,youcanexpecttogetthesamevalue back.Thisseemsonlynatural.Itmaybehardtoacceptatfirst,butwhenthereadsandwritesoccurindifferentthreads, thisissimplynotthecase.Ingeneral,thereisnoguaranteethatthereadingthreadwillseeavaluewrittenbyanother thread on a timely basis, or even at all. In order to ensure visibility of memory writes across threads, you must use synchronization.
NoVisibility in Listing 3.1 illustrates what can go wrong when threads share data without synchronization. Two threads,themainthreadandthereaderthread,accessthesharedvariablesreadyandnumber.Themainthreadstarts thereaderthreadandthensetsnumberto42andreadytotrue.Thereaderthreadspinsuntilitseesreadyistrue,and thenprintsoutnumber.WhileitmayseemobviousthatNoVisibilitywillprint42,itisinfactpossiblethatitwillprint zero,orneverterminateatall!Becauseitdoesnotuseadequatesynchronization,thereisnoguaranteethatthevalues ofreadyandnumberwrittenbythemainthreadwillbevisibletothereaderthread.
Listing3.1.SharingVariableswithoutSynchronization.
public class NoVisibility {
private static boolean ready;
private static int number;
private static class ReaderThread extends Thread {
public void run() {
while (!ready)
Thread.yield();
System.out.println(number);
}
}
public static void main(String[] args) {
new ReaderThread().start();
number = 42;
ready = true;
}
}
NoVisibility could loop forever because the value of ready might never become visible to the reader thread. Even morestrangely,NoVisibilitycouldprintzerobecausethewritetoreadymightbemadevisibletothereaderthread beforethewritetonumber,aphenomenonknownasreordering.Thereisnoguaranteethatoperationsinonethread willbeperformedintheordergivenbytheprogram,aslongasthereorderingisnotdetectablefromwithinthatthread Ͳevenifthereorderingisapparenttootherthreads.[1]Whenthemainthreadwritesfirsttonumberandthentodone withoutsynchronization,thereaderthreadcouldseethosewriteshappenintheoppositeorderͲornotatall.
[1] This may seem like a broken design, but it is meant to allow JVMs to take full advantage of the performance of modern multiprocessor hardware.Forexample,intheabsenceofsynchronization,theJavaMemoryModelpermitsthecompilertoreorderoperationsandcachevaluesin registers,andpermitsCPUstoreorderoperationsandcachevaluesinprocessorͲspecificcaches.Formoredetails,seeChapter16.
24 JavaConcurrencyInPractice
In the absence of synchronization, the compiler, processor, and runtime can do some downright weird things to the order in which operations appear to execute. Attempts to reason about the order in which memory actions "must"
happenininsufficientlysynchronizedmultithreadedprogramswillalmostcertainlybeincorrect.
NoVisibilityisaboutassimpleasaconcurrentprogramcangetͲtwothreadsandtwosharedvariablesͲandyetitis still all too easy to come to the wrong conclusions about what it does or even whether it will terminate. Reasoning aboutinsufficientlysynchronizedconcurrentprogramsisprohibitivelydifficult.
Thismayallsoundalittlescary,anditshould.Fortunately,there'saneasywaytoavoidthesecomplexissues:always usethepropersynchronizationwheneverdataissharedacrossthreads.
3.1.1.StaleData
NoVisibility demonstrated one of the ways that insufficiently synchronized programs can cause surprising results: stale data. When the reader thread examines ready, it may see an outͲofͲdate value. Unless synchronization is used every time a variable is accessed, it is possible to see a stale value for that variable. Worse, staleness is not allͲorͲ
nothing:athreadcanseeanupͲtoͲdatevalueofonevariablebutastalevalueofanothervariablethatwaswrittenfirst.
Whenfoodisstale,itisusuallystilledibleͲjustlessenjoyable.Butstaledatacanbemoredangerous.WhileanoutͲofͲ
datehitcounterinawebapplicationmightnotbesobad,[2]stalevaluescancauseserioussafetyorlivenessfailures.In NoVisibility,stalevaluescouldcauseittoprintthewrongvalueorpreventtheprogramfromterminating.Thingscan get even more complicated with stale values of object references, such as the link pointers in a linked list implementation. Stale data can cause serious and confusing failures such as unexpected exceptions, corrupted data structures,inaccuratecomputations,andinfiniteloops.
[2]ReadingdatawithoutsynchronizationisanalogoustousingtheREAD_UNCOMMITTEDisolationlevelinadatabase,whereyouarewillingto tradeaccuracyforperformance.However,inthecaseofunsynchronizedreads,youaretradingawayagreaterdegreeofaccuracy,sincethevisible valueforasharedvariablecanbearbitrarilystale.
MutableInteger in Listing 3.2 is not threadͲsafe because the value field is accessed from both get and set without synchronization.Amongotherhazards,itissusceptibletostalevalues:ifonethreadcallsset,otherthreadscallingget mayormaynotseethatupdate.
WecanmakeMutableIntegerthreadsafebysynchronizingthegetterandsetterasshowninSynchronizedIntegerin Listing 3.3. Synchronizing only the setter would not be sufficient: threads calling get would still be able to see stale values.
Listing3.2.NonǦthreadǦsafeMutableIntegerHolder.
@NotThreadSafe
public class MutableInteger {
private int value;
public int get() { return value; }
public void set(int value) { this.value = value; }
}
Listing3.3.ThreadǦsafeMutableIntegerHolder.
@ThreadSafe
public class SynchronizedInteger {
@GuardedBy("this") private int value;
public synchronized int get() { return value; }
public synchronized void set(int value) { this.value = value; }
}
3.1.2.NonǦatomic64ǦbitOperations
Whenathreadreadsavariablewithoutsynchronization,itmayseeastalevalue,butatleastitseesavaluethatwas actually placed there by some thread rather than some random value. This safety guarantee is called outͲofͲthinͲair safety.
OutͲofͲthinͲairsafetyappliestoallvariables,withoneexception:64Ͳbitnumericvariables(doubleandlong)thatare notdeclaredvolatile(seeSection3.1.4).TheJavaMemoryModelrequiresfetchandstoreoperationstobeatomic, butfornonvolatilelonganddoublevariables,theJVMispermittedtotreata64Ͳbitreadorwriteastwoseparate32Ͳ
bitoperations.Ifthereadsandwritesoccurindifferentthreads,itisthereforepossibletoreadanonvolatilelongand
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
25
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
getbackthehigh32bitsofonevalueandthelow32bitsofanother.[3]Thus,evenifyoudon'tcareaboutstalevalues,it is not safe to use shared mutable long and double variables in multithreaded programs unless they are declared volatileorguardedbyalock.
[3]WhentheJavaVirtualMachineSpecificationwaswritten,manywidelyusedprocessorarchitecturescouldnotefficientlyprovideatomic64Ͳbit arithmeticoperations.
3.1.3.LockingandVisibility
Intrinsic locking can be used to guarantee that one thread sees the effects of another in a predictable manner, as illustrated by Figure 3.1. When thread A executes a synchronized block, and subsequently thread B enters a synchronizedblockguardedbythesamelock,thevaluesofvariablesthatwerevisibletoApriortoreleasingthelock areguaranteedtobevisibletoBuponacquiringthelock.Inotherwords,everythingAdidinorpriortoasynchronized blockisvisibletoBwhenitexecutesasynchronizedblockguardedbythesamelock.Withoutsynchronization,thereis nosuchguarantee.
Figure3.1.VisibilityGuaranteesforSynchronization.
Wecannowgivetheotherreasonfortherulerequiringallthreadstosynchronizeonthesamelockwhenaccessinga sharedmutablevariableͲtoguaranteethatvalueswrittenbyonethreadaremadevisibletootherthreads.Otherwise, ifathreadreadsavariablewithoutholdingtheappropriatelock,itmightseeastalevalue.
Lockingisnotjustaboutmutualexclusion;itisalsoaboutmemoryvisibility.ToensurethatallthreadsseethemostupͲ
toͲdatevaluesofsharedmutablevariables,thereadingandwritingthreadsmustsynchronizeonacommonlock.
3.1.4.VolatileVariables
The Java language also provides an alternative, weaker form of synchronization, volatile variables, to ensure that updatestoavariablearepropagatedpredictablytootherthreads.Whenafieldisdeclaredvolatile,thecompilerand runtime are put on notice that this variable is shared and that operations on it should not be reordered with other memory operations. Volatile variables are not cached in registers or in caches where they are hidden from other processors,soareadofavolatilevariablealwaysreturnsthemostrecentwritebyanythread.
AgoodwaytothinkaboutvolatilevariablesistoimaginethattheybehaveroughlyliketheSynchronizedIntegerclass in Listing 3.3, replacing reads and writes of the volatile variable with calls to get and set.[4] Yet accessing a volatile variable performs no locking and so cannot cause the executing thread to block, making volatile variables a lighterͲ
weightsynchronizationmechanismthansynchronized.[5]
26 JavaConcurrencyInPractice
[4]Thisanalogyisnotexact;thememoryvisibilityeffectsofSynchronizedIntegerareactuallyslightlystrongerthanthoseofvolatilevariables.See Chapter16.
[5]Volatilereadsareonlyslightlymoreexpensivethannonvolatilereadsonmostcurrentprocessorarchitectures.
Thevisibilityeffectsofvolatilevariablesextendbeyondthevalueofthevolatilevariableitself.WhenthreadAwritesto avolatilevariableandsubsequentlythreadBreadsthatsamevariable,thevaluesofallvariablesthatwerevisibletoA priortowritingtothevolatilevariablebecomevisibletoBafterreadingthevolatilevariable.Sofromamemoryvisibility perspective,writingavolatilevariableislikeexitingasynchronizedblockandreadingavolatilevariableislikeentering asynchronizedblock.However,wedonotrecommendrelyingtooheavilyonvolatilevariablesforvisibility;codethat reliesonvolatilevariablesforvisibilityofarbitrarystateismorefragileandhardertounderstandthancodethatuses locking.
Use volatile variables only when they simplify implementing and verifying your synchronization policy; avoid using volatilevariableswhenverifyingcorrectnesswouldrequiresubtlereasoningaboutvisibility.Goodusesofvolatile variables include ensuring the visibility of their own state, that of the object they refer to, or indicating that an importantlifecycleevent(suchasinitializationorshutdown)hasoccurred.
Listing3.4illustratesatypicaluseofvolatilevariables:checkingastatusflagtodeterminewhentoexitaloop.Inthis example,ouranthropomorphizedthreadistryingtogettosleepbythetimeͲhonoredmethodofcountingsheep.For thisexampletowork,theasleepflagmustbevolatile.Otherwise,thethreadmightnotnoticewhenasleephasbeen setbyanotherthread.[6]Wecouldinsteadhaveusedlockingtoensurevisibilityofchangestoasleep,butthatwould havemadethecodemorecumbersome.
[6]Debuggingtip:Forserverapplications,besuretoalwaysspecifythe-serverJVMcommandlineswitchwheninvokingtheJVM,evenfor developmentandtesting.TheserverJVMperformsmoreoptimizationthantheclientJVM,suchashoistingvariablesoutofaloopthatarenot modified in the loop; code that might appear to work in the development environment (client JVM) can break in the deployment environment (serverJVM).Forexample,hadwe"forgotten"todeclarethevariableasleepasvolatileinListing3.4,theserverJVMcouldhoistthetestout oftheloop(turningitintoaninfiniteloop),buttheclientJVMwouldnot.Aninfiniteloopthatshowsupindevelopmentisfarlesscostlythanone thatonlyshowsupinproduction.
Listing3.4.CountingSheep.
volatile boolean asleep;
...
while (!asleep)
countSomeSheep();
Volatile variables are convenient, but they have limitations. The most common use for volatile variables is as a completion,interruption,orstatusflag,suchastheasleepflaginListing3.4.Volatilevariablescanbeusedforother kindsofstateinformation,butmorecareisrequiredwhenattemptingthis.Forexample,thesemanticsofvolatileare not strong enough to make the increment operation (count++) atomic, unless you can guarantee that the variable is writtenonlyfromasinglethread.(AtomicvariablesdoprovideatomicreadͲmodifyͲwritesupportandcanoftenbeused as"bettervolatilevariables";seeChapter15.)
Lockingcanguaranteebothvisibilityandatomicity;volatilevariablescanonlyguaranteevisibility.
Youcanusevolatilevariablesonlywhenallthefollowingcriteriaaremet:
x Writes to the variable do not depend on its current value, or you can ensure that only a single thread ever updatesthevalue;
x Thevariabledoesnotparticipateininvariantswithotherstatevariables;and x Lockingisnotrequiredforanyotherreasonwhilethevariableisbeingaccessed.
3.2.PublicationandEscape
Publishinganobjectmeansmakingitavailabletocodeoutsideofitscurrentscope,suchasbystoringareferencetoit whereothercodecanfindit,returningitfromanonͲprivatemethod,orpassingittoamethodinanotherclass.Inmany situations, we want to ensure that objects and their internals are not published. In other situations, we do want to publishanobjectforgeneraluse,butdoingsoinathreadͲsafemannermayrequiresynchronization.Publishinginternal state variables can compromise encapsulation and make it more difficult to preserve invariants; publishing objects beforethey arefullyconstructed cancompromisethreadsafety.Anobjectthatispublishedwhenitshouldnothave beenissaidtohaveescaped.Section3.5coversidiomsforsafepublication;rightnow,welookathowanobjectcan escape.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
27
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
Themostblatantformofpublicationistostoreareferenceinapublicstaticfield,whereanyclassandthreadcouldsee it,asinListing3.5.TheinitializemethodinstantiatesanewHashSetandpublishesitbystoringareferencetoitinto knownSecrets.
Listing3.5.PublishinganObject.
public static Set<Secret> knownSecrets;
public void initialize() {
knownSecrets = new HashSet<Secret>();
}
Publishingoneobjectmayindirectlypublishothers.IfyouaddaSecrettothepublishedknownSecretsset,you'vealso published that Secret, because any code can iterate the Set and obtain a reference to the new Secret. Similarly, returning a reference from a nonͲprivate method also publishes the returned object. UnsafeStates in Listing 3.6
publishesthesupposedlyprivatearrayofstateabbreviations.
Listing3.6.AllowingInternalMutableStatetoEscape.
class UnsafeStates {
private String[] states = new String[] {
"AK", "AL" ...
};
public String[] getStates() { return states; }
}
Publishingstatesinthiswayisproblematicbecauseanycallercanmodifyitscontents.Inthiscase,thestatesarray hasescapeditsintendedscope,becausewhatwassupposedtobeprivatestatehasbeeneffectivelymadepublic.
PublishinganobjectalsopublishesanyobjectsreferredtobyitsnonͲprivatefields.Moregenerally,anyobjectthatis reachable from a published object by following some chain of nonͲprivate field references and method calls has also beenpublished.
From the perspective of a class C, an alien method is one whose behavior is not fully specified by C. This includes methodsinotherclassesaswellasoverrideablemethods(neitherprivatenorfinal)inCitself.Passinganobjecttoan alienmethodmustalsobeconsideredpublishingthatobject.Sinceyoucan'tknowwhatcodewillactuallybeinvoked, youdon'tknowthatthealienmethodwon'tpublishtheobjectorretainareferencetoitthatmightlaterbeusedfrom anotherthread.
Whetheranotherthreadactuallydoessomethingwithapublishedreferencedoesn'treallymatter,becausetheriskof misuseisstillpresent.[7]Onceanobjectescapes,youhavetoassumethatanotherclassorthreadmay,maliciouslyor carelessly, misuse it. This is a compelling reason to use encapsulation: it makes it practical to analyze programs for correctnessandhardertoviolatedesignconstraintsaccidentally.
[7]Ifsomeonestealsyourpasswordandpostsitonthealt.freeͲpasswordsnewsgroup,thatinformationhasescaped:whetherornotsomeonehas (yet)usedthosecredentialstocreatemischief,youraccounthasstillbeencompromised.Publishingareferenceposesthesamesortofrisk.
A final mechanism by which an object or its internal state can be published is to publish an inner class instance, as shown in ThisEscape in Listing 3.7. When ThisEscape publishes the EventListener, it implicitly publishes the enclosing ThisEscape instance as well, because inner class instances contain a hidden reference to the enclosing instance.
28 JavaConcurrencyInPractice
Listing3.7.ImplicitlyAllowingthethisReferencetoEscape.
public class ThisEscape {
public ThisEscape(EventSource source) {
source.registerListener(
new EventListener() {
public void onEvent(Event e) {
doSomething(e);
}
});
}
}
3.2.1.SafeConstructionPractices
ThisEscape illustrates an important special case of escape Ͳ when the this references escapes during construction.
When the inner EventListener instance is published, so is the enclosing ThisEscape instance. But an object is in a predictable,consistentstateonlyafteritsconstructorreturns,sopublishinganobjectfromwithinitsconstructorcan publishanincompletelyconstructedobject.Thisistrueevenifthepublicationisthelaststatementintheconstructor.If thethisreferenceescapesduringconstruction,theobjectisconsiderednotproperlyconstructed.[8]
[8]Morespecifically,thethisreferenceshouldnotescapefromthethreaduntilaftertheconstructorreturns.Thethisreferencecanbestored somewherebytheconstructoraslongasitisnotusedbyanotherthreaduntilafterconstruction.SafeListenerinListing3.8usesthistechnique.
Donotallowthethisreferencetoescapeduringconstruction.
Acommonmistakethatcanletthethisreferenceescapeduringconstructionistostartathreadfromaconstructor.
Whenanobjectcreatesathreadfromitsconstructor,italmostalwayssharesitsthisreferencewiththenewthread, eitherexplicitly(bypassingittotheconstructor)orimplicitly(becausetheThreadorRunnableisaninnerclassofthe owning object). The new thread might then be able to see the owning object before it is fully constructed. There's nothingwrongwithcreatingathreadinaconstructor,butitisbestnottostartthethreadimmediately.Instead,expose a start or initialize method that starts the owned thread. (See Chapter 7 for more on service lifecycle issues.) Callinganoverrideableinstancemethod(onethatisneitherprivatenorfinal)fromtheconstructorcanalsoallowthe thisreferencetoescape.
If you are tempted to register an event listener or start a thread from a constructor, you can avoid the improper constructionbyusingaprivateconstructorandapublicfactorymethod,asshowninSafeListenerinListing3.8.
Listing3.8.UsingaFactoryMethodtoPreventthethisReferencefromEscapingDuringConstruction.
public class SafeListener {
private final EventListener listener;
private SafeListener() {
listener = new EventListener() {
public void onEvent(Event e) {
doSomething(e);
}
};
}
public static SafeListener newInstance(EventSource source) {
SafeListener safe = new SafeListener();
source.registerListener(safe.listener);
return safe;
}
}
3.3.ThreadConfinement
Accessing shared, mutable data requires using synchronization; one way to avoid this requirement is to not share. If dataisonlyaccessedfromasinglethread,nosynchronizationisneeded.Thistechnique,threadconfinement,isoneof thesimplestwaystoachievethreadsafety.Whenanobjectisconfinedtoathread,suchusageisautomaticallythreadͲ
safeeveniftheconfinedobjectitselfisnot[CPJ2.3.2].
Swingusesthreadconfinementextensively.TheSwingvisualcomponentsanddatamodelobjectsarenotthreadsafe; instead,safetyisachievedbyconfiningthemtotheSwingeventdispatchthread.TouseSwingproperly,coderunningin threads other than the event thread should not access these objects. (To make this easier, Swing provides the
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
29
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
invokeLatermechanismtoscheduleaRunnableforexecutionintheeventthread.)ManyconcurrencyerrorsinSwing applicationsstemfromimproperuseoftheseconfinedobjectsfromanotherthread.
AnothercommonapplicationofthreadconfinementistheuseofpooledJDBC(JavaDatabaseConnectivity)Connection objects.TheJDBCspecificationdoesnotrequirethatConnectionobjectsbethreadͲsafe.[9]Intypicalserverapplications, athreadacquiresaconnectionfromthepool,usesitforprocessingasinglerequest,andreturnsit.Sincemostrequests, such as servlet requests or EJB (Enterprise JavaBeans) calls, are processed synchronously by a single thread, and the pool will not dispense the same connection to another thread until it has been returned, this pattern of connection managementimplicitlyconfinestheConnectiontothatthreadforthedurationoftherequest.
[9]TheconnectionpoolimplementationsprovidedbyapplicationserversarethreadͲsafe;connectionpoolsarenecessarilyaccessedfrommultiple threads,soanonͲthreadͲsafeimplementationwouldnotmakesense.
Justasthelanguagehasnomechanismforenforcingthatavariableisguardedbyalock,ithasnomeansofconfiningan object to a thread. Thread confinement is an element of your program's design that must be enforced by its implementation.ThelanguageandcorelibrariesprovidemechanismsthatcanhelpinmaintainingthreadconfinementͲ
localvariablesandtheThreadLocalclassͲbutevenwiththese,itisstilltheprogrammer'sresponsibilitytoensurethat threadͲconfinedobjectsdonotescapefromtheirintendedthread.
3.3.1.AdǦhocThreadConfinement
AdͲhocthreadconfinementdescribeswhentheresponsibilityformaintainingthreadconfinementfallsentirelyonthe implementation. AdͲhoc thread confinement can be fragile because none of the language features, such as visibility modifiersorlocalvariables,helpsconfinetheobjecttothetargetthread.Infact,referencestothreadͲconfinedobjects suchasvisualcomponentsordatamodelsinGUIapplicationsareoftenheldinpublicfields.
Thedecision tousethreadconfinementisoftenaconsequenceofthedecisiontoimplementaparticularsubsystem, suchastheGUI,asasingleͲthreadedsubsystem.SingleͲthreadedsubsystemscansometimesofferasimplicitybenefit thatoutweighsthefragilityofadͲhocthreadconfinement.[10]
[10] Another reason to make a subsystem singleͲthreaded is deadlock avoidance; this is one of the primary reasons most GUI frameworks are singleͲthreaded.SingleͲthreadedsubsystemsarecoveredinChapter9.
Aspecialcaseofthreadconfinementappliestovolatilevariables.ItissafetoperformreadͲmodifyͲwriteoperationson sharedvolatilevariablesaslongasyouensurethatthevolatilevariableisonlywrittenfromasinglethread.Inthiscase, youareconfiningthemodificationtoasinglethreadtopreventraceconditions,andthevisibilityguaranteesforvolatile variablesensurethatotherthreadsseethemostupͲtoͲdatevalue.
Becauseofitsfragility,adͲhocthreadconfinementshouldbeusedsparingly;ifpossible,useoneofthestrongerformsof threadconfinement(stackconfinementorThreadLocal)instead.
3.3.2.StackConfinement
Stack confinement is a special case of thread confinement in which an object can only be reached through local variables.Justasencapsulationcanmakeiteasiertopreserveinvariants,localvariablescanmakeiteasiertoconfine objects to a thread. Local variables are intrinsically confined to the executing thread; they exist on the executing thread's stack, which is not accessible to other threads. Stack confinement (also called withinͲthread or threadͲlocal usage, but not to be confused with the ThreadLocal library class) is simpler to maintain and less fragile than adͲhoc threadconfinement.
For primitively typed local variables, such as numPairs in loadTheArk in Listing 3.9, you cannot violate stack confinementevenifyoutried.Thereisnowaytoobtainareferencetoaprimitivevariable,sothelanguagesemantics ensurethatprimitivelocalvariablesarealwaysstackconfined.
30 JavaConcurrencyInPractice
Listing3.9.ThreadConfinementofLocalPrimitiveandReferenceVariables.
public int loadTheArk(Collection<Animal> candidates) {
SortedSet<Animal> animals;
int numPairs = 0;
Animal candidate = null;
// animals confined to method, don't let them escape!
animals = new TreeSet<Animal>(new SpeciesGenderComparator()); animals.addAll(candidates);
for (Animal a : animals) {
if (candidate == null || !candidate.isPotentialMate(a))
candidate = a;
else {
ark.load(new AnimalPair(candidate, a));
++numPairs;
candidate = null;
}
}
return numPairs;
}
Maintainingstackconfinementforobjectreferencesrequiresalittlemoreassistancefromtheprogrammertoensure thatthereferentdoesnotescape.InloadTheArk,weinstantiateatreeSetandstoreareferencetoitinanimals.At this point, there is exactly one reference to the Set, held in a local variable and therefore confined to the executing thread. However, if we were to publish a reference to the Set (or any of its internals), the confinement would be violatedandtheanimalswouldescape.
Using a nonͲthreadͲsafe object in a withinͲthread context is still threadͲsafe. However, be careful: the design requirement that the object be confined to the executing thread, or the awareness that the confined object is not threadͲsafe,oftenexistsonlyintheheadofthedeveloperwhenthecodeiswritten.IftheassumptionofwithinͲthread usageisnotclearlydocumented,futuremaintainersmightmistakenlyallowtheobjecttoescape.
3.3.3.ThreadLocal
AmoreformalmeansofmaintainingthreadconfinementisThreadLocal,whichallowsyoutoassociateaperͲthread valuewithavalueͲholdingobject.Thread-Localprovidesgetandsetaccessormethodsthatmaintainaseparatecopy of the value for each thread that uses it, so a get returns the most recent value passed to set from the currently executingthread.
ThreadͲlocalvariablesareoftenusedtopreventsharingindesignsbasedonmutableSingletonsorglobalvariables.For example,asingleͲthreadedapplicationmightmaintainaglobaldatabaseconnectionthatisinitializedatstartuptoavoid having to pass a Connection to every method. Since JDBC connections may not be threadͲsafe, a multithreaded application that uses a global connection without additional coordination is not threadͲsafe either. By using a ThreadLocal to store the JDBC connection, as in ConnectionHolder in Listing 3.10, each thread will have its own connection.
Listing3.10.UsingThreadLocaltoEnsurethreadConfinement.
private static ThreadLocal<Connection> connectionHolder
= new ThreadLocal<Connection>() {
public Connection initialValue() {
return DriverManager.getConnection(DB_URL);
}
};
public static Connection getConnection() {
return connectionHolder.get();
}
This technique can also be used when a frequently used operation requires a temporary object such as a buffer and wantstoavoidreallocatingthetemporaryobjectoneachinvocation.Forexample,beforeJava5.0,Integer.toString usedaThreadLocaltostorethe12Ͳbytebufferusedforformattingitsresult,ratherthanusingasharedstaticbuffer (whichwouldrequirelocking)orallocatinganewbufferforeachinvocation.[11]
[11]Thistechniqueisunlikelytobeaperformancewinunlesstheoperationisperformedveryfrequentlyortheallocationisunusuallyexpensive.
InJava5.0,itwasreplacedwiththemorestraightforwardapproachofallocatinganewbufferforeveryinvocation,suggestingthatforsomething asmundaneasatemporarybuffer,itisnotaperformancewin.
WhenathreadcallsThreadLocal.getforthefirsttime,initialValueisconsultedtoprovidetheinitialvalueforthat thread. Conceptually, you can think of aThreadLocal<T> as holding a Map<Thread,T> that stores the threadͲspecific values, though this is not how it is actually implemented. The threadͲspecific values are stored in the Thread object itself;whenthethreadterminates,thethreadͲspecificvaluescanbegarbagecollected.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
31
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
If you are porting a singleͲthreaded application to a multithreaded environment, you can preserve thread safety by convertingsharedglobalvariablesintoThreadLocals,ifthesemanticsofthesharedglobalspermitsthis;anapplicationͲ
widecachewouldnotbeasusefulifitwereturnedintoanumberofthreadͲlocalcaches.
ThreadLocal is widely used in implementing application frameworks. For example, J2EE containers associate a transaction context with an executing thread for the duration of an EJB call. This is easily implemented using a static Thread-Localholdingthetransactioncontext:whenframeworkcodeneedstodeterminewhattransactioniscurrently running,itfetchesthetransactioncontextfromthisThreadLocal.Thisisconvenientinthatitreducestheneedtopass executioncontextinformationintoeverymethod,butcouplesanycodethatusesthismechanismtotheframework.
ItiseasytoabuseThreadLocalbytreatingitsthreadconfinementpropertyasalicensetouseglobalvariablesorasa meansofcreating"hidden"methodarguments.Likeglobalvariables,threadͲlocalvariablescandetractfromreusability andintroducehiddencouplingsamongclasses,andshouldthereforebeusedwithcare.
3.4.Immutability
TheotherendͲrunaroundtheneedtosynchronizeistouseimmutableobjects[EJItem13].Nearlyalltheatomicityand visibilityhazardswe'vedescribedsofar,suchasseeingstalevalues,losingupdates,orobservinganobjecttobeinan inconsistentstate,havetodowiththevagariesofmultiplethreadstryingtoaccessthesamemutablestateatthesame time.Ifanobject'sstatecannotbemodified,theserisksandcomplexitiessimplygoaway.
An immutable object is one whose state cannot be changed after construction. Immutable objects are inherently threadͲsafe;theirinvariantsareestablishedbytheconstructor,andiftheirstatecannotbechanged,theseinvariants alwayshold.
ImmutableobjectsarealwaysthreadͲsafe.
Immutableobjectsaresimple.Theycanonlybeinonestate,whichiscarefullycontrolledbytheconstructor.Oneofthe mostdifficultelementsofprogramdesignisreasoningaboutthepossiblestatesofcomplexobjects.Reasoningabout thestateofimmutableobjects,ontheotherhand,istrivial.
Immutable objects are also safer. Passing a mutable object to untrusted code, or otherwise publishing it where untrustedcodecouldfindit,isdangerousͲtheuntrustedcodemightmodifyitsstate,or,worse,retainareferencetoit and modify its state later from another thread. On the other hand, immutable objects cannot be subverted in this mannerbymaliciousorbuggycode,sotheyaresafetoshareandpublishfreelywithouttheneedtomakedefensive copies[EJItem24].
NeithertheJavaLanguageSpecificationnortheJavaMemoryModelformallydefinesimmutability,butimmutabilityis notequivalenttosimplydeclaringallfieldsofanobjectfinal.Anobjectwhosefieldsareallfinalmaystillbemutable, sincefinalfieldscanholdreferencestomutableobjects.
Anobjectisimmutableif:
x Itsstatecannotbemodifiedafterconstruction;
x Allitsfieldsarefinal;[12]and
x Itisproperlyconstructed(thethisreferencedoesnotescapeduringconstruction).
[12]Itistechnicallypossibletohaveanimmutableobjectwithoutallfieldsbeingfinal.StringissuchaclassͲbutthisreliesondelicatereasoning aboutbenigndataracesthatrequiresadeepunderstandingoftheJavaMemoryModel.(Forthecurious:Stringlazilycomputesthehashcodethe firsttimehashCodeiscalledandcachesitinanonͲfinalfield,butthisworksonlybecausethatfieldcantakeononlyonenonͲdefaultvaluethatis thesameeverytimeitiscomputedbecauseitisderiveddeterministicallyfromimmutablestate.Don'ttrythisathome.) Immutable objects can still use mutable objects internally to manage their state, as illustrated by ThreeStooges in Listing3.11.WhiletheSetthatstoresthenamesismutable,thedesignofThreeStoogesmakesitimpossibletomodify thatSetafterconstruction.Thestoogesreferenceisfinal,soallobjectstateisreachedthroughafinalfield.Thelast requirement,properconstruction,iseasilymetsincetheconstructordoesnothingthatwouldcausethethisreference tobecomeaccessibletocodeotherthantheconstructoranditscaller.
32 JavaConcurrencyInPractice
Listing3.11.ImmutableClassBuiltOutofMutableUnderlyingObjects.
@Immutable
public final class ThreeStooges {
private final Set<String> stooges = new HashSet<String>(); public ThreeStooges() {
stooges.add("Moe");
stooges.add("Larry");
stooges.add("Curly");
}
public boolean isStooge(String name) {
return stooges.contains(name);
}
}
Becauseprogramstatechangesallthetime,youmightbetemptedtothinkthatimmutableobjectsareoflimiteduse, but this is not the case. There is a difference between an object being immutable and the reference to it being immutable.Programstatestoredinimmutableobjectscanstillbeupdatedby"replacing"immutableobjectswithanew instanceholdingnewstate;thenextsectionoffersanexampleofthistechnique.[13]
[13]Manydevelopersfearthatthisapproachwill createperformanceproblems,butthesefearsareusuallyunwarranted.Allocationischeaper thanyoumightthink,andimmutableobjectsofferadditionalperformanceadvantagessuchasreducedneedforlockingordefensivecopiesand reducedimpactongenerationalgarbagecollection.
3.4.1.FinalFields
Thefinalkeyword,amorelimitedversionoftheconstmechanismfromC++,supportstheconstructionofimmutable objects.Finalfieldscan'tbemodified(althoughtheobjectstheyrefertocanbemodifiediftheyaremutable),butthey alsohavespecialsemanticsundertheJavaMemoryModel.Itistheuseoffinalfieldsthatmakespossibletheguarantee of initialization safety (see Section 3.5.2) that lets immutable objects be freely accessed and shared without synchronization.
Even if an object is mutable, making some fields final can still simplify reasoning about its state, since limiting the mutability of an object restricts its set of possible states. An object that is "mostly immutable" but has one or two mutablestatevariablesisstillsimplerthanonethathasmanymutablevariables.Declaringfieldsfinalalsodocuments tomaintainersthatthesefieldsarenotexpectedtochange.
Justasitisagoodpracticetomakeallfieldsprivateunlesstheyneedgreatervisibility[EJItem12],itisagoodpractice tomakeallfieldsfinalunlesstheyneedtobemutable.
3.4.2.Example:UsingVolatiletoPublishImmutableObjects
In UnsafeCachingFactorizer on page 24,we tried to use two AtomicReferences to store the last number and last factors, but this was not threadͲsafe because we could not fetch or update the two related values atomically. Using volatile variables for these values would not be threadͲsafe for the same reason. However, immutable objects can sometimesprovideaweakformofatomicity.
The factoring servlet performs two operations that must be atomic: updating the cached result and conditionally fetching the cached factors if the cached number matches the requested number. Whenever a group of related data itemsmustbeactedonatomically,considercreatinganimmutableholderclassforthem,suchasOneValueCache[14]in Listing3.12.
[14]OneValueCachewouldn'tbeimmutablewithoutthecopyOfcallsintheconstructorandgetter.Arrays.copyOfwasaddedasaconveniencein Java6;clonewouldalsowork.
Raceconditionsinaccessingorupdatingmultiplerelatedvariablescanbeeliminatedbyusinganimmutableobjectto hold all the variables. With a mutable holder object, you would have to use locking to ensure atomicity; with an immutableone,onceathreadacquiresareferencetoit,itneedneverworryaboutanotherthreadmodifyingitsstate.If thevariablesaretobeupdated,anewholderobjectiscreated,butanythreadsworkingwiththepreviousholderstill seeitinaconsistentstate.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
33
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
Listing3.12.ImmutableHolderforCachingaNumberanditsFactors.
@Immutable
class OneValueCache {
private final BigInteger lastNumber;
private final BigInteger[] lastFactors;
public OneValueCache(BigInteger i,
BigInteger[] factors) {
lastNumber = i;
lastFactors = Arrays.copyOf(factors, factors.length);
}
public BigInteger[] getFactors(BigInteger i) {
if (lastNumber == null || !lastNumber.equals(i))
return null;
else
return Arrays.copyOf(lastFactors, lastFactors.length);
}
}
VolatileCachedFactorizer in Listing 3.13 uses a OneValueCache to store the cached number and factors. When a thread sets the volatile cache field to reference a new OneValueCache, the new cached data becomes immediately visibletootherthreads.
ThecacheͲrelatedoperationscannotinterferewitheachother because One-ValueCacheisimmutableandthecache field is accessed only once in each of the relevant code paths. This combination of an immutable holder object for multiple state variables related by an invariant, and a volatile reference used to ensure its timely visibility, allows VolatileCachedFactorizertobethreadͲsafeeventhoughitdoesnoexplicitlocking.
3.5.SafePublication
Sofarwehavefocusedonensuringthatanobjectnotbepublished,suchaswhenitissupposedtobeconfinedtoa threadorwithinanotherobject.Ofcourse,sometimeswedowanttoshareobjectsacrossthreads,andinthiscasewe must do so safely. Unfortunately, simply storing a reference to an object into a public field, as in Listing 3.14, is not enoughtopublishthatobjectsafely.
Listing3.13.CachingtheLastResultUsingaVolatileReferencetoanImmutableHolderObject.
@ThreadSafe
public class VolatileCachedFactorizer implements Servlet {
private volatile OneValueCache cache =
new OneValueCache(null, null);
public void service(ServletRequest req, ServletResponse resp) {
BigInteger i = extractFromRequest(req);
BigInteger[] factors = cache.getFactors(i);
if (factors == null) {
factors = factor(i);
cache = new OneValueCache(i, factors);
}
encodeIntoResponse(resp, factors);
}
}
Listing3.14.PublishinganObjectwithoutAdequateSynchronization.
// Unsafe publication
public Holder holder;
public void initialize() {
holder = new Holder(42);
}
YoumaybesurprisedathowbadlythisharmlessͲlookingexamplecouldfail.Becauseofvisibilityproblems,theHolder couldappeartoanotherthreadtobeinaninconsistentstate,eventhoughitsinvariantswereproperlyestablishedbyits constructor!Thisimproperpublicationcouldallowanotherthreadtoobserveapartiallyconstructedobject.
3.5.1.ImproperPublication:WhenGoodObjectsGoBad
You cannot rely on the integrity of partially constructed objects. An observing thread could see the object in an inconsistentstate,andthenlaterseeitsstatesuddenlychange,eventhoughithasnotbeenmodifiedsincepublication.
34 JavaConcurrencyInPractice
Infact,iftheHolderinListing3.15ispublishedusingtheunsafepublicationidiominListing3.14,andathreadother thanthepublishingthreadweretocallassertSanity,itcouldthrowAssertionError![15]
[15] The problem here is not the Holder class itself, but that the Holder is not properly published. However, Holder can be made immune to improperpublicationbydeclaringthenfieldtobefinal,whichwouldmakeHolderimmutable;seeSection3.5.2.
Listing3.15.ClassatRiskofFailureifNotProperlyPublished.
public class Holder {
private int n;
public Holder(int n) { this.n = n; }
public void assertSanity() {
if (n != n)
throw new AssertionError("This statement is false.");
}
}
BecausesynchronizationwasnotusedtomaketheHoldervisibletootherthreads,wesaytheHolderwasnotproperly published. Two things can go wrongwith improperly published objects. Other threads could see a stale value for the holderfield,andthusseeanullreferenceorotheroldervalueeventhoughavaluehasbeenplacedinholder.Butfar worse, other threads could see an up to date value for the holder reference, but stale values for the state of the Holder.[16]Tomakethingsevenlesspredictable,athreadmayseeastalevaluethefirsttimeitreadsafieldandthena moreupͲtoͲdatevaluethenexttime,whichiswhyassertSanitycanthrowAssertionError.
[16]Whileitmayseemthatfieldvaluessetin aconstructorarethefirstvalueswrittentothosefieldsandthereforethatthereareno"older"
values to see as stale values, the Object constructor first writes the default values to all fields before subclass constructors run. It is therefore possibletoseethedefaultvalueforafieldasastalevalue.
At the risk of repeating ourselves, some very strange things can happen when data is shared across threads without sufficientsynchronization.
3.5.2.ImmutableObjectsandInitializationSafety
Becauseimmutableobjectsaresoimportant,theJavaMemoryModeloffersaspecialguaranteeofinitializationsafety for sharing immutable objects. As we've seen, that an object reference becomes visible to another thread does not necessarilymeanthatthestateofthatobjectisvisibletotheconsumingthread.Inordertoguaranteeaconsistentview oftheobject'sstate,synchronizationisneeded.
Immutable objects, on the other hand, can be safely accessed even when synchronization is not used to publish the objectreference.Forthisguaranteeofinitializationsafetytohold,alloftherequirementsforimmutabilitymustbemet: unmodifiable state, all fields are final, and proper construction. (If Holder in Listing 3.15 were immutable, assertSanitycouldnotthrowAssertionError,eveniftheHolderwasnotproperlypublished.) Immutableobjectscanbeusedsafelybyanythreadwithoutadditionalsynchronization,evenwhensynchronizationis notusedtopublishthem.
Thisguaranteeextendstothevaluesofallfinalfieldsofproperlyconstructedobjects;finalfieldscanbesafelyaccessed withoutadditionalsynchronization.However,iffinalfieldsrefertomutableobjects,synchronizationisstillrequiredto accessthestateoftheobjectstheyreferto.
3.5.3.SafePublicationIdioms
Objectsthatarenotimmutablemustbesafelypublished,whichusuallyentailssynchronizationbyboththepublishing andtheconsumingthread.Forthemoment,let'sfocusonensuringthattheconsumingthreadcanseetheobjectinits aspublishedstate;we'lldealwithvisibilityofmodificationsmadeafterpublicationsoon.
To publish an object safely, both the reference to the object and the object's state must be made visible to other threadsatthesametime.Aproperlyconstructedobjectcanbesafelypublishedby: x Initializinganobjectreferencefromastaticinitializer;
x StoringareferencetoitintoavolatilefieldorAtomicReference;
x Storingareferencetoitintoafinalfieldofaproperlyconstructedobject;or x Storingareferencetoitintoafieldthatisproperlyguardedbyalock.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
35
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects
TheinternalsynchronizationinthreadͲsafecollectionsmeansthatplacinganobjectinathreadͲsafecollection,suchasa Vector or synchronizedList, fulfills the last of these requirements. If thread A places object X in a threadͲsafe collection and thread B subsequently retrieves it, B is guaranteed to see the state of X as A left it, even though the applicationcodethathandsXoffinthismannerhasnoexplicitsynchronization.ThethreadͲsafelibrarycollectionsoffer thefollowingsafepublicationguarantees,eveniftheJavadocislessthanclearonthesubject: x PlacingakeyorvalueinaHashtable,synchronizedMap,orConcurrent-Mapsafelypublishesittoanythread thatretrievesitfromtheMap(whetherdirectlyorviaaniterator);
x Placing an element in a Vector, CopyOnWriteArrayList, CopyOnWrite-ArraySet, synchronizedList, or synchronizedSetsafelypublishesittoanythreadthatretrievesitfromthecollection; x Placing an element on a BlockingQueue or a ConcurrentLinkedQueue safely publishes it to any thread that retrievesitfromthequeue.
Otherhandoffmechanismsintheclasslibrary(suchasFutureandExchanger)alsoconstitutesafepublication;wewill identifytheseasprovidingsafepublicationastheyareintroduced.
Usingastaticinitializerisoftentheeasiestandsafestwaytopublishobjectsthatcanbestaticallyconstructed: public static Holder holder = new Holder(42);
StaticinitializersareexecutedbytheJVMatclassinitializationtime;becauseofinternalsynchronizationintheJVM,this mechanismisguaranteedtosafelypublishanyobjectsinitializedinthisway[JLS12.4.2].
3.5.4.EffectivelyImmutableObjects
Safepublicationissufficientforotherthreadstosafelyaccessobjectsthatarenotgoingtobemodifiedafterpublication without additional synchronization. The safe publication mechanisms all guarantee that the asͲpublished state of an object is visible to all accessing threads as soon as the reference to it is visible, and if that state is not going to be changedagain,thisissufficienttoensurethatanyaccessissafe.
Objectsthatarenottechnicallyimmutable,butwhosestatewillnotbemodifiedafterpublication,arecalledeffectively immutable.TheydonotneedtomeetthestrictdefinitionofimmutabilityinSection3.4;theymerelyneedtobetreated by the program as if they were immutable after they are published. Using effectively immutable objects can simplify developmentandimproveperformancebyreducingtheneedforsynchronization.
Safelypublishedeffectivelyimmutableobjectscanbeusedsafelybyanythreadwithoutadditionalsynchronization.
Forexample,Dateismutable,[17]butifyouuseitasifitwereimmutable,youmaybeabletoeliminatethelockingthat wouldotherwiseberequiredwhensharedaDateacrossthreads.SupposeyouwanttomaintainaMapstoringthelast logintimeofeachuser:
[17]Thiswasprobablyamistakeintheclasslibrarydesign.
public Map<String, Date> lastLogin =
Collections.synchronizedMap(new HashMap<String, Date>());
IftheDatevaluesarenotmodifiedaftertheyareplacedintheMap,thenthesynchronizationinthesynchronizedMap implementation is sufficient to publish the Date values safely, and no additional synchronization is needed when accessingthem.
3.5.5.MutableObjects
If an object may be modified after construction, safe publication ensures only the visibility of the asͲpublished state.
Synchronizationmustbeusednotonlytopublishamutableobject,butalsoeverytimetheobjectisaccessedtoensure visibility of subsequent modifications. To share mutable objects safely, they must be safely published and be either threadͲsafeorguardedbyalock.
Thepublicationrequirementsforanobjectdependonitsmutability:
x Immutableobjectscanbepublishedthroughanymechanism;
x Effectivelyimmutableobjectsmustbesafelypublished;
x Mutableobjectsmustbesafelypublished,andmustbeeitherthreadͲsafeorguardedbyalock.
36 JavaConcurrencyInPractice
3.5.6.SharingObjectsSafely
Wheneveryouacquireareferencetoanobject,youshouldknowwhatyouareallowedtodowithit.Doyouneedto acquirealockbeforeusingit?Areyouallowedtomodifyitsstate,oronlytoreadit?Manyconcurrencyerrorsstem from failing to understand these "rules of engagement" for a shared object. When you publish an object, you should documenthowtheobjectcanbeaccessed.
Themostusefulpoliciesforusingandsharingobjectsinaconcurrentprogramare: ThreadͲconfined.AthreadͲconfinedobjectisownedexclusivelybyandconfinedtoonethread,andcanbemodifiedby itsowningthread.
Shared readͲonly. A shared readͲonly object can be accessed concurrently by multiple threads without additional synchronization, but cannot be modified by any thread. Shared readͲonly objects include immutable and effectively immutableobjects.
Shared threadͲsafe. A threadͲsafe object performs synchronization internally, so multiple threads can freely access it throughitspublicinterfacewithoutfurthersynchronization.
Guarded. A guarded object can be accessed only with a specific lock held. Guarded objects include those that are encapsulatedwithinotherthreadͲsafeobjectsandpublishedobjectsthatareknowntobeguardedbyaspecificlock.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
37
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
Chapter4.ComposingObjects
Sofar,we'vecoveredthelowͲlevelbasicsofthreadsafetyandsynchronization.Butwedon'twanttohavetoanalyze eachmemoryaccesstoensurethatourprogramisthreadͲsafe;wewanttobeabletotakethreadͲsafecomponentsand safelycomposethemintolargercomponentsorprograms.Thischaptercoverspatternsforstructuringclassesthatcan make it easier to make them threadͲsafe and to maintain them without accidentally undermining their safety guarantees.
4.1.DesigningaThreadǦsafeClass
WhileitispossibletowriteathreadͲsafeprogramthatstoresallitsstateinpublicstaticfields,itisalothardertoverify its thread safety or to modify it so that it remains threadͲsafe than one that uses encapsulation appropriately.
EncapsulationmakesitpossibletodeterminethataclassisthreadͲsafewithouthavingtoexaminetheentireprogram.
ThedesignprocessforathreadͲsafeclassshouldincludethesethreebasicelements: x Identifythevariablesthatformtheobject'sstate;
x Identifytheinvariantsthatconstrainthestatevariables;
x Establishapolicyformanagingconcurrentaccesstotheobject'sstate.
Anobject'sstatestartswithitsfields.Iftheyareallofprimitivetype,thefieldscomprisetheentirestate.Counterin Listing4.1hasonlyonefield,sothevaluefieldcomprisesitsentirestate.Thestateofanobjectwithnprimitivefieldsis justthenͲtupleofitsfieldvalues;thestateofa2DPointisits(x,y)value.Iftheobjecthasfieldsthatarereferencesto other objects, its state will encompass fields from the referenced objects as well. For example, the state of a LinkedListincludesthestateofallthelinknodeobjectsbelongingtothelist.
ThesynchronizationpolicydefineshowanobjectcoordinatesaccesstoitsstatewithoutviolatingitsinvariantsorpostͲ
conditions.Itspecifieswhatcombinationofimmutability,threadconfinement,andlockingisusedtomaintainthread safety, and which variables are guarded by which locks. To ensure that the class can be analyzed and maintained, documentthesynchronizationpolicy.
Listing4.1.SimpleThreadǦsafeCounterUsingtheJavaMonitorPattern.
@ThreadSafe
public final class Counter {
@GuardedBy("this") private long value = 0;
public synchronized long getValue() {
return value;
}
public synchronized long increment() {
if (value == Long.MAX_VALUE)
throw new IllegalStateException("counter overflow");
return ++value;
}
}
4.1.1.GatheringSynchronizationRequirements
Making a class threadͲsafe means ensuring that its invariants hold under concurrent access; this requires reasoning aboutitsstate.Objectsandvariableshaveastatespace:therangeofpossiblestatestheycantakeon.Thesmallerthis statespace,theeasieritistoreasonabout.Byusingfinalfieldswhereverpractical,youmakeitsimplertoanalyzethe possiblestatesanobjectcanbein.(Intheextremecase,immutableobjectscanonlybeinasinglestate.) Many classeshaveinvariantsthatidentifycertainstatesasvalid orinvalid.Thevaluefieldin Counterisalong. The state space of a long ranges from Long.MIN_VALUE to Long.MAX_VALUE, but Counter places constraints on value; negativevaluesarenotallowed.
Similarly,operationsmayhavepostͲconditionsthatidentifycertainstatetransitionsasinvalid.Ifthecurrentstateofa Counter is 17, the only valid next state is 18. When the next state is derived from the current state, the operation is necessarily a compound action. Not all operations impose state transition constraints; when updating a variable that holdsthecurrenttemperature,itspreviousstatedoesnotaffectthecomputation.
ConstraintsplacedonstatesorstatetransitionsbyinvariantsandpostͲconditionscreateadditionalsynchronizationor encapsulation requirements. If certain states are invalid, then the underlying state variables must be encapsulated, otherwiseclientcodecouldputtheobjectintoaninvalidstate.Ifanoperationhasinvalidstatetransitions,itmustbe
38 JavaConcurrencyInPractice
made atomic. On the other hand, if the class does not impose any such constraints, we may be able to relax encapsulationorserializationrequirementstoobtaingreaterflexibilityorbetterperformance.
Aclasscanalsohaveinvariantsthatconstrainmultiplestatevariables.Anumberrangeclass,likeNumberRangeinListing 4.10, typically maintains state variables for the lower and upper bounds of the range. These variables must obey the constraintthatthelowerboundbelessthanorequaltotheupperbound.Multivariableinvariantslikethisonecreate atomicityrequirements:relatedvariablesmustbefetchedorupdatedinasingleatomicoperation.Youcannotupdate one,releaseandreacquirethelock,andthenupdatetheothers,sincethiscouldinvolveleavingtheobjectinaninvalid statewhenthelockwasreleased.Whenmultiplevariablesparticipateinaninvariant,thelockthatguardsthemmust beheldforthedurationofanyoperationthataccessestherelatedvariables.
Youcannotensurethreadsafetywithoutunderstandinganobject'sinvariantsandpostͲconditions.Constraintsonthe validvaluesorstatetransitionsforstatevariablescancreateatomicityandencapsulationrequirements.
4.1.2.StateǦdependentOperations
ClassinvariantsandmethodpostͲconditionsconstrainthevalidstatesandstatetransitionsforanobject.Someobjects alsohavemethodswithstateͲbasedpreconditions.Forexample,youcannotremoveanitemfromanemptyqueue;a queuemustbeinthe"nonempty"statebeforeyoucanremoveanelement.OperationswithstateͲbasedpreconditions arecalledstateͲdependent[CPJ3].
In a singleͲthreaded program, if a precondition does not hold, the operation has no choice but to fail. But in a concurrentprogram,thepreconditionmaybecometruelaterduetotheactionofanotherthread.Concurrentprograms addthepossibilityofwaitinguntilthepreconditionbecomestrue,andthenproceedingwiththeoperation.
ThebuiltͲinmechanismsforefficientlywaitingforaconditiontobecometrueͲwaitandnotifyͲaretightlyboundto intrinsiclocking,andcanbedifficulttousecorrectly.Tocreateoperationsthatwaitforapreconditiontobecometrue beforeproceeding,itisofteneasiertouseexistinglibraryclasses,suchasblockingqueuesorsemaphores,toprovide the desired stateͲdependent behavior. Blocking library classes such as BlockingQueue, Semaphore, and other synchronizersarecoveredinChapter5;creatingstateͲdependentclassesusingthelowͲlevelmechanismsprovidedby theplatformandclasslibraryiscoveredinChapter14.
4.1.3.StateOwnership
WeimpliedinSection4.1thatanobject'sstatecouldbeasubsetofthefieldsintheobjectgraphrootedatthatobject.
Whymightitbeasubset?Underwhatconditionsarefieldsreachablefromagivenobjectnotpartofthatobject'sstate?
Whendefiningwhichvariablesformanobject'sstate,wewanttoconsideronlythedatathatobjectowns.Ownershipis not embodied explicitly in the language, but is instead an element of class design. If you allocate and populate a HashMap, you are creating multiple objects: the HashMap object, a number of Map.Entry objects used by the implementationofHashMap,andperhapsotherinternalobjectsaswell.ThelogicalstateofaHashMapincludesthestate ofallitsMap.Entryandinternalobjects,eventhoughtheyareimplementedasseparateobjects.
Forbetterorworse,garbagecollectionletsusavoidthinkingcarefullyaboutownership.Whenpassinganobjecttoa method in C++, you have to think fairly carefully about whether you are transferring ownership, engaging in a shortͲ
term loan, or envisioning longͲterm joint ownership. In Java, all these same ownership models are possible, but the garbage collector reduces the cost of many of the common errors in reference sharing, enabling lessͲthanͲprecise thinkingaboutownership.
Inmanycases,ownershipandencapsulationgotogetherͲtheobjectencapsulatesthestateitownsandownsthestate itencapsulates.Itistheownerofagivenstatevariablethatgetstodecideonthelockingprotocolusedtomaintainthe integrityofthatvariable'sstate.Ownershipimpliescontrol,butonceyoupublishareferencetoamutableobject,you nolongerhaveexclusivecontrol;atbest,youmighthave"sharedownership".Aclassusuallydoesnotowntheobjects passedtoitsmethodsorconstructors,unlessthemethodisdesignedtoexplicitlytransferownershipofobjectspassed in(suchasthesynchronizedcollectionwrapperfactorymethods).
Collection classes often exhibit a form of "split ownership", in which the collection owns the state of the collection infrastructure,butclientcodeownstheobjectsstoredinthecollection.AnexampleisServletContextfromtheservlet framework. ServletContext provides a MapͲlike object container service to servlets where they can register and retrieveapplicationobjectsbynamewithsetAttributeandgetAttribute.TheServletContextobjectimplemented bytheservletcontainermustbethreadͲsafe,becauseitwillnecessarilybeaccessedbymultiplethreads.Servletsneed not use synchronization when calling set-Attribute and getAttribute, but they may have to use synchronization when using the objects stored in the ServletContext. These objects are owned by the application; they are being
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
39
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
storedforsafekeepingbytheservletcontainerontheapplication'sbehalf.Likeallsharedobjects,theymustbeshared safely; in order to prevent interference from multiple threads accessing the same object concurrently, they should eitherbethreadͲsafe,effectivelyimmutable,orexplicitlyguardedbyalock.[1]
[1]Interestingly,theHttpSessionobject,whichperformsasimilarfunctionintheservletframework,mayhavestricterrequirements.Because theservletcontainermayaccesstheobjectsintheHttpSessionsotheycanbeserializedforreplicationorpassivation,theymustbethreadͲ
safebecausethecontainerwillbeaccessingthemaswellasthewebapplication.(Wesay"mayhave"sincereplicationandpassivationisoutside oftheservletspecificationbutisacommonfeatureofservletcontainers.) 4.2.InstanceConfinement
If an object is not threadͲsafe, several techniques can still let it be used safely in a multithreaded program. You can ensurethatitisonlyaccessedfromasinglethread(threadconfinement),orthatallaccesstoitisproperlyguardedbya lock.
EncapsulationsimplifiesmakingclassesthreadͲsafebypromotinginstanceconfinement,oftenjustcalledconfinement
[CPJ2.3.3].Whenanobjectisencapsulatedwithinanotherobject,allcodepathsthathaveaccesstotheencapsulated object are known and can be therefore be analyzed more easily than if that object were accessible to the entire program. Combining confinement with an appropriate locking discipline can ensure that otherwise nonͲthreadͲsafe objectsareusedinathreadͲsafemanner.
Encapsulatingdatawithinanobjectconfinesaccesstothedatatotheobject'smethods,makingiteasiertoensurethat thedataisalwaysaccessedwiththeappropriatelockheld.
Confinedobjectsmustnotescapetheirintendedscope.Anobjectmaybeconfinedtoaclassinstance(suchasaprivate classmember),alexicalscope(suchasalocalvariable),orathread(suchasanobjectthatispassedfrommethodto methodwithinathread,butnotsupposedtobesharedacrossthreads).Objectsdon'tescapeontheirown,ofcourseͲ
theyneedhelpfromthedeveloper,whoassistsbypublishingtheobjectbeyonditsintendedscope.
PersonSet in Listing 4.2 illustrates how confinement and locking can work together to make a class threadͲsafe even whenitscomponentstatevariablesarenot.ThestateofPersonSetismanagedbyaHashSet,whichisnotthreadͲsafe.
ButbecausemySetisprivateandnotallowedtoescape,theHashSetisconfinedtothePersonSet.Theonlycodepaths thatcanaccessmySetareaddPersonandcontainsPerson,andeachoftheseacquiresthelockonthePersonSet.Allits stateisguardedbyitsintrinsiclock,makingPersonSetthreadͲsafe.
Listing4.2.UsingConfinementtoEnsureThreadSafety.
@ThreadSafe
public class PersonSet {
@GuardedBy("this")
private final Set<Person> mySet = new HashSet<Person>(); public synchronized void addPerson(Person p) {
mySet.add(p);
}
public synchronized boolean containsPerson(Person p) {
return mySet.contains(p);
}
}
ThisexamplemakesnoassumptionsaboutthethreadͲsafetyofPerson,butifitismutable,additionalsynchronization willbeneededwhenaccessingaPersonretrievedfromaPersonSet.Themostreliablewaytodothiswouldbetomake PersonthreadͲsafe;lessreliablewouldbetoguardthePersonobjectswithalockandensurethatallclientsfollowthe protocolofacquiringtheappropriatelockbeforeaccessingthePerson.
Instance confinement is one of the easiest ways to build threadͲsafe classes. It also allows flexibility in the choice of locking strategy;PersonSet happened to use its own intrinsic lock to guard its state, but any lock, consistently used, woulddojustaswell.Instanceconfinementalsoallowsdifferentstatevariablestobeguardedbydifferentlocks.(Foran exampleofaclassthatusesmultiplelockobjectstoguarditsstate,seeServerStatuson236.) Therearemanyexamplesofconfinementintheplatformclasslibraries,includingsomeclassesthatexistsolelytoturn nonͲthreadͲsafe classes into threadͲsafe ones. The basic collection classes such as ArrayList and HashMap are not threadͲsafe,buttheclasslibraryprovideswrapperfactorymethods(Collections.synchronizedListandfriends)so theycanbeusedsafelyinmultithreadedenvironments.ThesefactoriesusetheDecoratorpattern(Gammaetal.,1995) towrapthecollectionwithasynchronizedwrapperobject;thewrapperimplementseachmethodoftheappropriate
40 JavaConcurrencyInPractice
interface as a synchronized method that forwards the request to the underlying collection object. So long as the wrapperobjectholdstheonlyreachablereferencetotheunderlyingcollection(i.e.,theunderlyingcollectionisconfined to the wrapper), the wrapper object is then threadͲsafe. The Javadoc for these methods warns that all access to the underlyingcollectionmustbemadethroughthewrapper.
Ofcourse,itisstillpossibletoviolateconfinementbypublishingasupposedlyconfinedobject;ifanobjectisintended tobeconfinedtoaspecificscope,thenlettingitescapefromthatscopeisabug.Confinedobjectscanalsoescapeby publishingotherobjectssuchasiteratorsorinnerclassinstancesthatmayindirectlypublishtheconfinedobjects.
Confinement makes it easier to build threadͲsafe classes because a class that confines its state can be analyzed for threadsafetywithouthavingtoexaminethewholeprogram.
4.2.1.TheJavaMonitorPattern
Following the principle of instance confinement to its logical conclusion leads you to the Java monitor pattern.[2] An objectfollowingtheJavamonitorpatternencapsulatesallitsmutablestateandguardsitwiththeobject'sownintrinsic lock.
[2]TheJavamonitorpatternisinspiredbyHoare'sworkonmonitors(Hoare,1974),thoughtherearesignificantdifferencesbetweenthispattern andatruemonitor.Thebytecodeinstructionsforenteringandexitingasynchronizedblockareevencalledmonitorenterandmonitorexit,and Java'sbuiltͲin(intrinsic)locksaresometimescalledmonitorlocksormonitors.
CounterinListing4.1showsatypicalexampleofthispattern.Itencapsulatesonestatevariable,value,andallaccess tothatstatevariableisthroughthemethodsofCounter,whichareallsynchronized.
The Java monitor pattern is used by many library classes, such as Vector and Hashtable. Sometimes a more sophisticated synchronization policy is needed; Chapter 11 shows how to improve scalability through finerͲgrained lockingstrategies.TheprimaryadvantageoftheJavamonitorpatternisitssimplicity.
TheJavamonitorpatternismerelyaconvention;anylockobjectcouldbeusedtoguardanobject'sstatesolongasitis usedconsistently.Listing4.3illustratesaclassthatusesaprivatelocktoguarditsstate.
Listing4.3.GuardingStatewithaPrivateLock.
public class PrivateLock {
private final Object myLock = new Object();
@GuardedBy("myLock") Widget widget;
void someMethod() {
synchronized(myLock) {
// Access or modify the state of widget
}
}
}
Thereareadvantagestousingaprivatelockobjectinsteadofanobject'sintrinsiclock(oranyotherpubliclyaccessible lock). Making the lock object private encapsulates the lock so that client code cannot acquire it, whereas a publicly accessible lock allows client code to participate in its synchronization policy Ͳ correctly or incorrectly. Clients that improperlyacquireanotherobject'slockcouldcauselivenessproblems,andverifyingthatapubliclyaccessiblelockis properlyusedrequiresexaminingtheentireprogramratherthanasingleclass.
4.2.2.Example:TrackingFleetVehicles
Counter in Listing 4.1 is a concise, but trivial, example of the Java monitor pattern. Let's build a slightly less trivial example:a"vehicletracker"fordispatchingfleetvehiclessuchastaxicabs,policecars,ordeliverytrucks.We'llbuildit first using the monitor pattern, and then see how to relax some of the encapsulation requirements while retaining threadsafety.
EachvehicleisidentifiedbyaStringandhasalocationrepresentedby(x,y)coordinates.TheVehicleTrackerclasses encapsulatetheidentityandlocationsoftheknownvehicles,makingthemwellͲsuitedasadatamodelinamodelviewͲ
controller GUI application where it might be shared by a view thread and multiple updater threads. The view thread wouldfetchthenamesandlocationsofthevehiclesandrenderthemonadisplay: Map<String, Point> locations = vehicles.getLocations();
for (String key : locations.keySet())
renderVehicle(key, locations.get(key));
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
41
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
Similarly,theupdaterthreadswouldmodifyvehiclelocationswithdatareceivedfromGPSdevicesorenteredmanually byadispatcherthroughaGUIinterface:
void vehicleMoved(VehicleMovedEvent evt) {
Point loc = evt.getNewLocation();
vehicles.setLocation(evt.getVehicleId(), loc.x, loc.y);
}
Sincetheviewthreadandtheupdaterthreadswillaccessthedatamodelconcurrently,itmustbethreadͲsafe.Listing 4.4showsanimplementationofthevehicletrackerusingtheJavamonitorpatternthatusesMutablePointinListing4.5
forrepresentingthevehiclelocations.
Even though MutablePoint is not threadͲsafe, the tracker class is. Neither the map nor any of the mutable points it contains is ever published. When we need to a return vehicle locations to callers, the appropriate values are copied usingeithertheMutablePointcopyconstructorordeepCopy,whichcreatesanewMapwhosevaluesarecopiesofthe keysandvaluesfromtheoldMap.[3]
[3]NotethatdeepCopycan'tjustwraptheMapwithanunmodifiableMap,becausethatprotectsonlythecollectionfrommodification;itdoesnot preventcallersfrommodifyingthemutableobjectsstoredinit.Forthesamereason,populatingtheHashMapindeepCopyviaacopyconstructor wouldn'tworkeither,becauseonlythereferencestothepointswouldbecopied,notthepointobjectsthemselves.
Thisimplementation maintainsthread safetyinpartbycopying mutabledatabeforereturningitto theclient. Thisis usually not a performance issue, but could become one if the set of vehicles is very large.[4] Another consequence of copyingthedataoneachcalltogetLocationisthatthecontentsofthereturnedcollectiondonotchangeevenifthe underlyinglocationschange.Whetherthisisgoodorbaddependsonyourrequirements.Itcouldbeabenefitifthere areinternalconsistencyrequirementsonthelocationset,inwhichcasereturningaconsistentsnapshotiscritical,ora drawbackifcallersrequireupͲtoͲdateinformationforeachvehicleandthereforeneedtorefreshtheirsnapshotmore often.
[4]BecausedeepCopyiscalledfromasynchronizedmethod,thetracker'sintrinsiclockisheldforthedurationofwhatmightbealongͲrunning copyoperation,andthiscoulddegradetheresponsivenessoftheuserinterfacewhenmanyvehiclesarebeingtracked.
4.3.DelegatingThreadSafety
All but the most trivial objects are composite objects. The Java monitor pattern is useful when building classes from scratchorcomposingclassesoutofobjectsthatarenotthreadͲsafe.Butwhatifthecomponentsofourclassarealready threadͲsafe?Doweneedtoaddanadditionallayerofthreadsafety?Theansweris..."itdepends".Insomecasesa compositemadeofthreadͲsafecomponentsisthreadͲsafe(Listings4.7and4.9),andinothersitismerelyagoodstart (4.10).
In CountingFactorizer on page 23, we added an AtomicLong to an otherwise stateless object, and the resulting composite object was still threadͲsafe. Since the state of CountingFactorizer is the state of the threadͲsafe AtomicLong,andsinceCountingFactorizerimposesnoadditionalvalidityconstraintsonthestateofthecounter,itis easy to see that CountingFactorizer is threadͲsafe. We could say that CountingFactorizer delegates its thread safetyresponsibilitiestotheAtomicLong:CountingFactorizeristhreadͲsafebecauseAtomicLongis.[5]
[5]Ifcountwerenotfinal,thethreadsafetyanalysisofCountingFactorizerwouldbemorecomplicated.IfCountingFactorizercouldmodifycount toreferenceadifferentAtomicLong,wewouldthenhavetoensurethatthisupdatewasvisibletoallthreadsthatmightaccessthecount,andthat therewerenoraceconditionsregardingthevalueofthecountreference.Thisisanothergoodreasontousefinalfieldswhereverpractical.
42 JavaConcurrencyInPractice
Listing4.4.MonitorǦbasedVehicleTrackerImplementation.
@ThreadSafe
public class MonitorVehicleTracker {
@GuardedBy("this")
private final Map<String, MutablePoint> locations;
public MonitorVehicleTracker(
Map<String, MutablePoint> locations) {
this.locations = deepCopy(locations);
}
public synchronized Map<String, MutablePoint> getLocations() {
return deepCopy(locations);
}
public synchronized MutablePoint getLocation(String id) {
MutablePoint loc = locations.get(id);
return loc == null ? null : new MutablePoint(loc);
}
public synchronized void setLocation(String id, int x, int y) {
MutablePoint loc = locations.get(id);
if (loc == null)
throw new IllegalArgumentException("No such ID: " + id); loc.x = x;
loc.y = y;
}
private static Map<String, MutablePoint> deepCopy(
Map<String, MutablePoint> m) {
Map<String, MutablePoint> result =
new HashMap<String, MutablePoint>();
for (String id : m.keySet())
result.put(id, new MutablePoint(m.get(id)));
return Collections.unmodifiableMap(result);
}
}
public class MutablePoint { /* Listing 4.5 */ }
Listing4.5.MutablePointClassSimilartoJava.awt.Point.
@NotThreadSafe
public class MutablePoint {
public int x, y;
public MutablePoint() { x = 0; y = 0; }
public MutablePoint(MutablePoint p) {
this.x = p.x;
this.y = p.y;
}
}
4.3.1.Example:VehicleTrackerUsingDelegation
Asamoresubstantialexampleofdelegation,let'sconstructaversionofthevehicletrackerthatdelegatestoathreadͲ
safeclass.WestorethelocationsinaMap,sowestartwithathreadͲsafeMapimplementation,ConcurrentHashMap.We alsostorethelocationusinganimmutablePointclassinsteadofMutablePoint,showninListing4.6.
Listing4.6.ImmutablePointclassusedbyDelegatingVehicleTracker.
@Immutable
public class Point {
public final int x, y;
public Point(int x, int y) {
this.x = x;
this.y = y;
}
}
Point is threadͲsafe because it is immutable. Immutable values can be freely shared and published, so we no longer needtocopythelocationswhenreturningthem.
DelegatingVehicleTrackerinListing4.7doesnotuseanyexplicitsynchronization;allaccesstostateismanagedby ConcurrentHashMap,andallthekeysandvaluesoftheMapareimmutable.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
43
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
If we had used the original MutablePoint class instead of Point, we would be breaking encapsulation by letting getLocationspublishareferencetomutablestatethatisnotthreadͲsafe.Noticethatwe'vechangedthebehaviorof thevehicletrackerclassslightly;whilethemonitorversionreturnedasnapshotofthelocations,thedelegatingversion returns an unmodifiable but "live" view of the vehicle locations. This means that if thread A calls getLocations and threadBlatermodifiesthelocationofsomeofthepoints,thosechangesarereflectedintheMapreturnedtothreadA.
Asweremarkedearlier,thiscanbeabenefit(moreupͲtoͲdatedata)oraliability(potentiallyinconsistentviewofthe fleet),dependingonyourrequirements.
Ifanunchangingviewofthefleetisrequired,getLocationscouldinsteadreturnashallowcopyofthelocationsmap.
SincethecontentsoftheMapareimmutable,onlythestructureoftheMap,notthecontents,mustbecopied,asshown inListing4.8(whichreturnsaplainHashMap,sincegetLocationsdidnotpromisetoreturnathreadͲsafeMap).
Listing4.7.DelegatingThreadSafetytoaConcurrentHashMap.
@ThreadSafe
public class DelegatingVehicleTracker {
private final ConcurrentMap<String, Point> locations;
private final Map<String, Point> unmodifiableMap;
public DelegatingVehicleTracker(Map<String, Point> points) {
locations = new ConcurrentHashMap<String, Point>(points);
unmodifiableMap = Collections.unmodifiableMap(locations);
}
public Map<String, Point> getLocations() {
return unmodifiableMap;
}
public Point getLocation(String id) {
return locations.get(id);
}
public void setLocation(String id, int x, int y) {
if (locations.replace(id, new Point(x, y)) == null)
throw new IllegalArgumentException(
"invalid vehicle name: " + id);
}
}
Listing4.8.ReturningaStaticCopyoftheLocationSetInsteadofa"Live"One.
public Map<String, Point> getLocations() {
return Collections.unmodifiableMap(
new HashMap<String, Point>(locations));
}
4.3.2.IndependentStateVariables
Thedelegationexamplessofardelegatetoasingle,threadͲsafestatevariable.Wecanalsodelegatethreadsafetyto morethanoneunderlyingstatevariableaslongasthoseunderlyingstatevariablesareindependent,meaningthatthe compositeclassdoesnotimposeanyinvariantsinvolvingthemultiplestatevariables.
VisualComponent in Listing 4.9 is a graphical component that allows clients to register listeners for mouse and keystrokeevents.Itmaintainsalistofregisteredlistenersofeachtype,sothatwhenaneventoccurstheappropriate listenerscanbeinvoked.Butthereisnorelationshipbetweenthesetofmouselistenersandkeylisteners;thetwoare independent,andthereforeVisualComponentcandelegateitsthreadsafetyobligationstotwounderlyingthreadͲsafe lists.
VisualComponentusesaCopyOnWriteArrayListtostoreeachlistenerlist;thisisathreadͲsafeListimplementation particularly suited for managing listener lists (see Section 5.2.3). Each List is threadͲsafe, and because there are no constraints coupling the state of one to the state of the other, VisualComponent can delegate its thread safety responsibilitiestotheunderlyingmouseListenersandkeyListenersobjects.
44 JavaConcurrencyInPractice
Listing4.9.DelegatingThreadSafetytoMultipleUnderlyingStateVariables.
public class VisualComponent {
private final List<KeyListener> keyListeners
= new CopyOnWriteArrayList<KeyListener>();
private final List<MouseListener> mouseListeners
= new CopyOnWriteArrayList<MouseListener>();
public void addKeyListener(KeyListener listener) {
keyListeners.add(listener);
}
public void addMouseListener(MouseListener listener) {
mouseListeners.add(listener);
}
public void removeKeyListener(KeyListener listener) {
keyListeners.remove(listener);
}
public void removeMouseListener(MouseListener listener) {
mouseListeners.remove(listener);
}
}
4.3.3.WhenDelegationFails
MostcompositeclassesarenotassimpleasVisualComponent:theyhaveinvariantsthatrelatetheircomponentstate variables. NumberRange in Listing 4.10 uses two AtomicIntegers to manage its state, but imposes an additional constraintͲthatthefirstnumberbelessthanorequaltothesecond.
NumberRangeisnotthreadͲsafe;itdoesnotpreservetheinvariantthatconstrainslowerandupper.ThesetLowerand setUppermethodsattempttorespectthisinvariant,butdosopoorly.BothsetLowerandsetUpperarecheckͲthenͲact sequences, but they do not use sufficient locking to make them atomic. If the number range holds (0, 10), and one threadcallssetLower(5)whileanotherthreadcallssetUpper(4),withsomeunluckytimingbothwillpassthechecks inthesettersand both modifications willbeapplied.Theresultisthattherangenow holds(5,4)aninvalidstate. So whiletheunderlyingAtomicIntegersarethreadͲsafe,thecompositeclassisnot.Becausetheunderlyingstatevariables lower and upper are not independent, NumberRange cannot simply delegate thread safety to its threadͲsafe state variables.
NumberRangecouldbemadethreadͲsafebyusinglockingtomaintainitsinvariants,suchasguardinglowerandupper withacommonlock.Itmustalsoavoidpublishingloweranduppertopreventclientsfromsubvertingitsinvariants.
If a class has compound actions, as NumberRange does, delegation alone is again not a suitable approach for thread safety.Inthesecases,theclassmustprovideitsownlockingtoensurethatcompoundactionsareatomic,unlessthe entirecompoundactioncanalsobedelegatedtotheunderlyingstatevariables.
IfaclassiscomposedofmultipleindependentthreadͲsafestatevariablesandhasnooperationsthathaveanyinvalid statetransitions,thenitcandelegatethreadsafetytotheunderlyingstatevariables.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
45
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
Listing4.10.NumberRangeClassthatdoesNotSufficientlyProtectItsInvariants.
public class NumberRange {
// INVARIANT: lower <= upper
private final AtomicInteger lower = new AtomicInteger(0);
private final AtomicInteger upper = new AtomicInteger(0);
public void setLower(int i) {
// Warning -- unsafe check-then-act
if (i > upper.get())
throw new IllegalArgumentException(
"can't set lower to " + i + " > upper");
lower.set(i);
}
public void setUpper(int i) {
// Warning -- unsafe check-then-act
if (i < lower.get())
throw new IllegalArgumentException(
"can't set upper to " + i + " < lower");
upper.set(i);
}
public boolean isInRange(int i) {
return (i >= lower.get() && i <= upper.get());
}
}
TheproblemthatpreventedNumberRangefrombeingthreadͲsafeeventhoughitsstatecomponentswerethreadͲsafeis very similar to one of the rules about volatile variables described in Section 3.1.4: a variable is suitable for being declaredvolatileonlyifitdoesnotparticipateininvariantsinvolvingotherstatevariables.
4.3.4.PublishingUnderlyingStateVariables
Whenyoudelegatethreadsafetytoanobject'sunderlyingstatevariables,underwhatconditionscanyoupublishthose variables so that other classes can modify them as well? Again, the answer depends on what invariants your class imposes on those variables. While the underlying value field in Counter could take on any integer value, Counter constrainsittotakeononlypositivevalues,andtheincrementoperationconstrainsthesetofvalidnextstatesgiven anycurrentstate.Ifyouweretomakethevaluefieldpublic,clientscouldchangeittoaninvalidvalue,sopublishingit wouldrendertheclassincorrect.Ontheotherhand,ifavariablerepresentsthecurrenttemperatureortheIDofthe lastusertologon,thenhavinganotherclassmodifythisvalueatanytimeprobablywouldnotviolateanyinvariants,so publishing this variable might be acceptable. (It still may not be a good idea, since publishing mutable variables constrains future development and opportunities for subclassing, but it would not necessarily render the class not threadͲsafe.)
IfastatevariableisthreadͲsafe,doesnotparticipateinanyinvariantsthatconstrainitsvalue,andhasnoprohibited statetransitionsforanyofitsoperations,thenitcansafelybepublished.
For example, it would be safe to publish mouseListeners or keyListeners in VisualComponent. Because VisualComponent does not impose any constraints on the valid states of its listener lists, these fields could be made publicorotherwisepublishedwithoutcompromisingthreadsafety.
4.3.5.Example:VehicleTrackerthatPublishesItsState
Let's construct another version of the vehicle tracker that publishes its underlying mutable state. Again, we need to modifytheinterfacealittlebittoaccommodatethischange,thistimeusingmutablebutthreadͲsafepoints.
46 JavaConcurrencyInPractice
Listing4.11.ThreadǦsafeMutablePointClass.
@ThreadSafe
public class SafePoint {
@GuardedBy("this") private int x, y;
private SafePoint(int[] a) { this(a[0], a[1]); }
public SafePoint(SafePoint p) { this(p.get()); }
public SafePoint(int x, int y) {
this.x = x;
this.y = y;
}
public synchronized int[] get() {
return new int[] { x, y };
}
public synchronized void set(int x, int y) {
this.x = x;
this.y = y;
}
}
SafePointinListing4.11providesagetterthatretrievesboththexandyvaluesatoncebyreturningatwoͲelement array.[6]Ifweprovidedseparategettersforxandy,thenthevaluescouldchangebetweenthetimeonecoordinateis retrievedandtheother,resultinginacallerseeinganinconsistentvalue:an(x,y)locationwherethevehicleneverwas.
UsingSafePoint,wecanconstructavehicletrackerthatpublishestheunderlyingmutablestatewithoutundermining threadsafety,asshowninthePublishingVehicleTrackerclassinListing4.12.
[6]Theprivateconstructorexiststoavoidtheraceconditionthatwouldoccurifthecopyconstructorwereimplementedasthis(p.x,p.y);thisisan exampleoftheprivateconstructorcaptureidiom(BlochandGafter,2005).
PublishingVehicleTracker derives its thread safety from delegation to an underlying ConcurrentHashMap, but this time the contents of the Map are threadͲsafe mutable points rather than immutable ones. The getLocation method returns an unmodifiable copy of the underlying Map. Callers cannot add or remove vehicles, but could change the locationofoneofthevehiclesbymutatingtheSafePointvaluesinthereturnedMap.Again,the"live"natureoftheMap maybeabenefitoradrawback,dependingontherequirements.PublishingVehicleTrackeristhreadͲsafe,butwould not be so if it imposed any additional constraints on the valid values for vehicle locations. If it needed to be able to
"veto" changes to vehicle locations or to take action when a location changes, the approach taken by PublishingVehicleTrackerwouldnotbeappropriate.
Listing4.12.VehicleTrackerthatSafelyPublishesUnderlyingState.
@ThreadSafe
public class PublishingVehicleTracker {
private final Map<String, SafePoint> locations;
private final Map<String, SafePoint> unmodifiableMap;
public PublishingVehicleTracker(
Map<String, SafePoint> locations) {
this.locations
= new ConcurrentHashMap<String, SafePoint>(locations);
this.unmodifiableMap
= Collections.unmodifiableMap(this.locations);
}
public Map<String, SafePoint> getLocations() {
return unmodifiableMap;
}
public SafePoint getLocation(String id) {
return locations.get(id);
}
public void setLocation(String id, int x, int y) {
if (!locations.containsKey(id))
throw new IllegalArgumentException(
"invalid vehicle name: " + id);
locations.get(id).set(x, y);
}
}
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
47
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
4.4.AddingFunctionalitytoExistingThreadǦsafeClasses
The Java class library contains many useful "building block" classes. Reusing existing classes is often preferable to creating new ones: reuse can reduce development effort, development risk (because the existing components are already tested), and maintenance cost. Sometimes a threadͲsafe class that supports all of the operations we want alreadyexists,butoftenthebestwecanfindisaclassthatsupportsalmostalltheoperationswewant,andthenwe needtoaddanewoperationtoitwithoutunderminingitsthreadsafety.
As an example, let's say we need a threadͲsafe List with an atomic putͲifͲabsent operation. The synchronized List implementationsnearlydothejob,sincetheyprovidethecontainsandaddmethodsfromwhichwecanconstructa putͲifͲabsentoperation.
TheconceptofputͲifͲabsentisstraightforwardenoughͲchecktoseeifanelementisinthecollectionbeforeaddingit, anddonotadditifitisalreadythere.(Your"checkͲthenͲact"warningbellsshouldbegoingoffnow.)Therequirement that the class be threadͲsafe implicitly adds another requirement Ͳ that operations like putͲifͲabsent be atomic. Any reasonableinterpretationsuggeststhat,ifyoutakeaListthatdoesnotcontainobjectX,andaddXtwicewithputͲifͲ
absent,theresultingcollectioncontainsonlyonecopyofX.But,ifputͲifͲabsentwerenotatomic,withsomeunlucky timingtwothreadscouldbothseethatXwasnotpresentandbothaddX,resultingintwocopiesofX.
Thesafestwaytoaddanewatomicoperationistomodifytheoriginalclasstosupportthedesiredoperation,butthisis notalwayspossiblebecauseyoumaynothaveaccesstothesource codeormaynotbe freetomodifyit.Ifyou can modifytheoriginalclass,youneedtounderstandtheimplementation'ssynchronizationpolicysothatyoucanenhance itinamannerconsistentwithitsoriginaldesign.Addingthenewmethoddirectlytotheclassmeansthatallthecode that implements the synchronization policy for that class is still contained in one source file, facilitating easier comprehensionandmaintenance.
Anotherapproachistoextendtheclass,assumingitwasdesignedforextension.BetterVectorinListing4.13extends VectortoaddaputIfAbsentmethod.ExtendingVectorisstraightforwardenough,butnotallclassesexposeenough oftheirstatetosubclassestoadmitthisapproach.
Extensionismorefragilethanaddingcodedirectlytoaclass,becausetheimplementationofthesynchronizationpolicy is now distributed over multiple, separately maintained source files. If the underlying class were to change its synchronization policy by choosing a different lock to guard its state variables, the subclass would subtly and silently break, because it no longer used the right lock to control concurrent access to the base class's state. (The synchronizationpolicyofVectorisfixedbyitsspecification,soBetterVectorwouldnotsufferfromthisproblem.) Listing4.13.ExtendingVectortohaveaPutǦifǦabsentMethod.
@ThreadSafe
public class BetterVector<E> extends Vector<E> {
public synchronized boolean putIfAbsent(E x) {
boolean absent = !contains(x);
if (absent)
add(x);
return absent;
}
}
4.4.1.ClientǦsideLocking
ForanArrayListwrappedwithaCollections.synchronizedListwrapper,neitheroftheseapproachesͲadding a methodtotheoriginalclassorextendingtheclassͲworksbecausetheclientcodedoesnotevenknowtheclassofthe Listobjectreturnedfromthesynchronizedwrapperfactories.Athirdstrategyistoextendthefunctionalityoftheclass withoutextendingtheclassitselfbyplacingextensioncodeina"helper"class.
Listing4.14showsafailedattempttocreateahelperclasswithanatomicputͲifͲabsentoperationforoperatingona threadͲsafeList.
48 JavaConcurrencyInPractice
Listing4.14.NonǦthreadǦsafeAttempttoImplementPutǦifǦabsent.
@NotThreadSafe
public class ListHelper<E> {
public List<E> list =
Collections.synchronizedList(new ArrayList<E>());
...
public synchronized boolean putIfAbsent(E x) {
boolean absent = !list.contains(x);
if (absent)
list.add(x);
return absent;
}
}
Why wouldn't this work? After all, putIfAbsent is synchronized, right? The problem is that it synchronizes on the wrong lock. Whatever lock the List uses to guard its state, it sure isn't the lock on the ListHelper. ListHelper provides only the illusion of synchronization; the various list operations, while all synchronized, use different locks, whichmeansthatputIfAbsentisnotatomicrelativetootheroperationsonthe List.So thereisnoguaranteethat anotherthreadwon'tmodifythelistwhileputIfAbsentisexecuting.
Tomakethisapproachwork,wehavetousethesamelockthattheListusesbyusingclientͲsidelockingorexternal locking.ClientͲsidelockingentailsguardingclient codethatuses someobjectXwiththelockXusestoguarditsown state.InordertouseclientͲsidelocking,youmustknowwhatlockXuses.
ThedocumentationforVectorandthesynchronizedwrapperclassesstates,albeitobliquely,thattheysupportclientͲ
side locking, by using the intrinsic lock for the Vector or the wrapper collection (not the wrapped collection). Listing 4.15showsaputIfAbsentoperationonathreadͲsafeListthatcorrectlyusesclientͲsidelocking.
Listing4.15.ImplementingPutǦifǦabsentwithClientǦsideLocking.
@ThreadSafe
public class ListHelper<E> {
public List<E> list =
Collections.synchronizedList(new ArrayList<E>());
...
public boolean putIfAbsent(E x) {
synchronized (list) {
boolean absent = !list.contains(x);
if (absent)
list.add(x);
return absent;
}
}
}
If extending a class to add another atomic operation is fragile because it distributes the locking code for a class over multipleclassesinanobjecthierarchy,clientͲsidelockingisevenmorefragilebecauseitentailsputtinglockingcodefor classCintoclassesthataretotallyunrelatedtoC. Exercisecarewhenusing clientͲsidelockingonclassesthatdonot committotheirlockingstrategy.
ClientͲsidelockinghasalotincommonwithclassextensionͲtheybothcouplethebehaviorofthederivedclasstothe implementationofthebaseclass.Justasextensionviolatesencapsulationofimplementation[EJItem 14],clientͲside lockingviolatesencapsulationofsynchronizationpolicy.
4.4.2.Composition
There is a less fragile alternative for adding an atomic operation to an existing class: composition. ImprovedList in Listing 4.16 implements the List operations by delegating them to an underlying List instance, and adds an atomic putIfAbsentmethod.(LikeCollections.synchronizedListandother collectionswrappers,ImprovedListassumes that once a list is passed to its constructor, the client will not use the underlying list directly again, accessing it only throughtheImprovedList.)
ImprovedList adds an additional level of locking using its own intrinsic lock. It does not care whether the underlying List is threadͲsafe, because it provides its own consistent locking that provides thread safety even if the List is not threadͲsafe or changes its locking implementation. While the extra layer of synchronization may add some small performance penalty,[7] the implementation in ImprovedList is less fragile than attempting to mimic the locking
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
49
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4.
ComposingObjects
strategyofanotherobject.Ineffect,we'veusedtheJavamonitorpatterntoencapsulateanexistingList,andthisis guaranteedtoprovidethreadsafetysolongasourclassholdstheonlyoutstandingreferencetotheunderlyingList.
Listing4.16.ImplementingPutǦifǦabsentUsingComposition.
@ThreadSafe
public class ImprovedList<T> implements List<T> {
private final List<T> list;
public ImprovedList(List<T> list) { this.list = list; }
public synchronized boolean putIfAbsent(T x) {
boolean contains = list.contains(x);
if (contains)
list.add(x);
return !contains;
}
public synchronized void clear() { list.clear(); }
// ... similarly delegate other List methods
}
[7]ThepenaltywillbesmallbecausethesynchronizationontheunderlyingListisguaranteedtobeuncontendedandthereforefast;seeChapter 11.
4.5.DocumentingSynchronizationPolicies
Documentation is one of the most powerful (and, sadly, most underutilized) tools for managing thread safety. Users looktothedocumentationtofindoutifaclassisthreadͲsafe,andmaintainerslooktothedocumentationtounderstand theimplementationstrategysotheycanmaintainitwithoutinadvertentlycompromisingsafety.Unfortunately,bothof theseconstituenciesusuallyfindlessinformationinthedocumentationthanthey'dlike.
Documentaclass'sthreadsafetyguaranteesforitsclients;documentitssynchronizationpolicyforitsmaintainers.
Eachuseofsynchronized,volatile,oranythreadͲsafeclassreflectsasynchronizationpolicydefiningastrategyfor ensuringtheintegrityofdatainthefaceofconcurrentaccess.Thatpolicyisanelementofyourprogram'sdesign,and shouldbedocumented.Ofcourse,thebesttimetodocumentdesigndecisionsisatdesigntime.Weeksormonthslater, thedetailsmaybeablurͲsowriteitdownbeforeyouforget.
Craftingasynchronizationpolicyrequiresanumberofdecisions:whichvariablestomakevolatile,whichvariablesto guardwithlocks,whichlock(s)guardwhichvariables,whichvariablestomakeimmutableorconfinetoathread,which operationsmustbeatomic,etc.Someofthesearestrictlyimplementationdetailsandshouldbedocumentedforthe sake of future maintainers, but some affect the publicly observable locking behavior of your class and should be documentedaspartofitsspecification.
Attheveryleast,documentthethreadsafetyguaranteesmadebyaclass.IsitthreadͲsafe?Doesitmakecallbackswith alockheld?Arethereanyspecificlocksthataffectitsbehavior?Don'tforceclientstomakeriskyguesses.Ifyoudon't want to commit to supporting clientͲside locking, that's fine, but say so. If you want clients to be able to create new atomicoperationsonyourclass,aswedidinSection4.4,youneedtodocumentwhichlockstheyshouldacquiretodo sosafely.Ifyouuselockstoguardstate,documentthisforfuturemaintainers,becauseit'ssoeasyͲthe@GuardedBy annotationwilldothetrick.Ifyouusemoresubtlemeanstomaintainthreadsafety,documentthembecausetheymay notbeobvioustomaintainers.
Thecurrentstateofaffairsinthreadsafetydocumentation,evenintheplatformlibraryclasses,isnotencouraging.How manytimeshaveyoulookedattheJavadocforaclassandwonderedwhetheritwasthreadͲsafe?[8]Mostclassesdon't offer any clue either way. Many official Java technology specifications, such as servlets and JDBC, woefully underdocumenttheirthreadsafetypromisesandrequirements.
[8]Ifyou'veneverwonderedthis,weadmireyouroptimism.
Whileprudencesuggeststhatwenotassumebehaviorsthataren'tpartofthespecification,wehaveworktogetdone, andweareoftenfacedwithachoiceofbadassumptions.ShouldweassumeanobjectisthreadͲsafebecauseitseems thatitoughttobe?ShouldweassumethataccesstoanobjectcanbemadethreadͲsafebyacquiringitslockfirst?(This riskytechniqueworksonlyifwecontrolallthecodethataccessesthatobject;otherwise,itprovidesonlytheillusionof threadsafety.)Neitherchoiceisverysatisfying.
50 JavaConcurrencyInPractice
Tomakemattersworse,ourintuitionmayoftenbewrongonwhichclassesare"probablythreadͲsafe"andwhichare not. As an example, java.text.SimpleDateFormat isn't threadͲsafe, but the Javadoc neglected to mention this until JDK 1.4. That this particular class isn't threadͲsafe comes as a surprise to many developers. How many programs mistakenly createasharedinstanceofanonͲthreadͲsafeobjectandusedit frommultiplethreads,unawarethatthis mightcauseerroneousresultsunderheavyload?
TheproblemwithSimpleDateFormatcouldbeavoidedbynotassumingaclassisthreadͲsafeifitdoesn'tsayso.Onthe other hand, it is impossible to develop a servletͲbased application without making some pretty questionable assumptions about the thread safety of containerͲprovided objects like HttpSession. Don't make your customers or colleagueshavetomakeguesseslikethis.
4.5.1.InterpretingVagueDocumentation
Many Java technology specifications are silent, or at least unforthcoming, about thread safety guarantees and requirements for interfaces such as ServletContext, HttpSession, or DataSource.[9] Since these interfaces are implementedbyyourcontainerordatabasevendor,youoftencan'tlookatthecodetoseewhatitdoes.Besides,you don't want to rely on the implementation details of one particular JDBC driver Ͳ you want to be compliant with the standardsoyourcodeworksproperlywithanyJDBCdriver.Butthewords"thread"and"concurrent"donotappearat allintheJDBCspecification,andappearfrustratinglyrarelyintheservletspecification.Sowhatdoyoudo?
[9]Wefinditparticularlyfrustratingthattheseomissionspersistdespitemultiplemajorrevisionsofthespecifications.
Youaregoingtohavetoguess.Onewaytoimprovethequalityofyourguessistointerpretthespecificationfromthe perspectiveofsomeonewhowillimplementit(suchasacontainerordatabasevendor),asopposedtosomeonewho willmerelyuseit.ServletsarealwayscalledfromacontainerͲmanagedthread,anditissafetoassumethatifthereis morethanonesuchthread,thecontainerknowsthis.Theservletcontainermakesavailablecertainobjectsthatprovide servicetomultipleservlets,suchasHttpSessionorServletContext.Sotheservletcontainershouldexpecttohave these objects accessed concurrently, since it has created multiple threads and called methods like Servlet.service fromthemthatcouldreasonablybeexpectedtoaccesstheServletContext.
SinceitisimpossibletoimagineasingleͲthreadedcontextinwhichtheseobjectswouldbeuseful,onehastoassume that they have been made threadͲsafe, even though the specification does not explicitly require this. Besides, if they required clientͲside locking, on what lock should the client code synchronize? The documentation doesn't say, and it seems absurd to guess. This "reasonable assumption" is further bolstered by the examples in the specification and official tutorials that show how to access ServletContext or HttpSession and do not use any clientͲside synchronization.
Ontheotherhand,theobjectsplacedintheServletContextorHttpSessionwithsetAttributeareownedbythe webapplication,nottheservletcontainer.Theservletspecificationdoesnotsuggestanymechanismforcoordinating concurrentaccesstosharedattributes.Soattributesstoredbythecontaineronbehalfofthewebapplicationshouldbe threadͲsafeoreffectivelyimmutable.Ifallthecontainerdidwasstoretheseattributesonbehalfofthewebapplication, anotheroptionwouldbetoensurethattheyareconsistentlyguardedbyalockwhenaccessedfromservletapplication code. But because the container may want to serialize objects in the HttpSession for replication or passivation purposes,andtheservletcontainercan'tpossiblyknowyourlockingprotocol,youshouldmakethemthreadͲsafe.
OnecanmakeasimilarinferenceabouttheJDBCDataSourceinterface,whichrepresentsapoolofreusabledatabase connections. A DataSource provides service to an application, and it doesn't make much sense in the context of a singleͲthreadedapplication.Itishardtoimagineausecasethatdoesn'tinvolvecallinggetConnectionfrommultiple threads.And,aswithservlets,theexamplesintheJDBCspecificationdonotsuggesttheneedforanyclientͲsidelocking in the many code examples using DataSource. So, even though the specification doesn't promise that DataSource is threadͲsafeorrequirecontainervendorstoprovideathreadͲsafeimplementation,bythesame"itwouldbeabsurdifit weren't" argument, we have no choice but to assume that DataSource.getConnection does not require additional clientͲsidelocking.
On the other hand, we would not make the same argument about the JDBC Connection objects dispensed by the DataSource,sincethesearenotnecessarilyintendedtobesharedbyotheractivitiesuntiltheyarereturnedtothepool.
So if an activity that obtains a JDBC Connection spans multiple threads, it must take responsibility for ensuring that access to the Connection is properly guarded by synchronization. (In most applications, activities that use a JDBC
ConnectionareimplementedsoastoconfinetheConnectiontoaspecificthreadanyway.)
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
51
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
Chapter5.BuildingBlocks
ThelastchapterexploredseveraltechniquesforconstructingthreadͲsafeclasses,includingdelegatingthreadsafetyto existingthreadͲsafeclasses.Wherepractical,delegationisoneofthemosteffectivestrategiesforcreatingthreadͲsafe classes:justletexistingthreadͲsafeclassesmanageallthestate.
Theplatformlibrariesincludearichsetofconcurrentbuildingblocks,suchasthreadͲsafecollectionsandavarietyof synchronizers that can coordinate the control flow of cooperating threads. This chapter covers the most useful concurrent building blocks, especially those introduced in Java 5.0 and Java 6, and some patterns for using them to structureconcurrentapplications.
5.1.SynchronizedCollections
The synchronized collection classes include Vector and Hashtable, part of the original JDK, as well as their cousins added in JDK 1.2, the synchronized wrapper classes created by the Collections.synchronizedXxx factory methods.
Theseclassesachievethreadsafetybyencapsulatingtheirstateandsynchronizingeverypublicmethodsothatonlyone threadatatimecanaccessthecollectionstate.
5.1.1.ProblemswithSynchronizedCollections
ThesynchronizedcollectionsarethreadͲsafe,butyoumaysometimesneedtouseadditionalclientͲsidelockingtoguard compound actions. Common compound actions on collections include iteration (repeatedly fetch elements until the collection is exhausted), navigation (find the next element after this one according to some order), and conditional operations such as putͲifͲabsent (check if a Map has a mapping for key K, and if not, add the mapping (K,V)). With a synchronized collection, these compound actions are still technically threadͲsafe even without clientͲside locking, but theymaynotbehaveasyoumightexpectwhenotherthreadscanconcurrentlymodifythecollection.
Listing5.1showstwomethodsthatoperateonaVector,getLastanddelete-Last,bothofwhicharecheckͲthenͲact sequences.Eachcallssizetodeterminethesizeofthearrayandusestheresultingvaluetoretrieveorremovethelast element.
Listing5.1.CompoundActionsonaVectorthatmayProduceConfusingResults.
public static Object getLast(Vector list) {
int lastIndex = list.size() - 1;
return list.get(lastIndex);
}
public static void deleteLast(Vector list) {
int lastIndex = list.size() - 1;
list.remove(lastIndex);
}
Thesemethodsseemharmless,andinasensetheyareͲtheycan'tcorrupttheVector,nomatterhowmanythreads callthemsimultaneously.Butthecallerofthesemethodsmighthaveadifferentopinion.IfthreadAcallsgetLastona Vectorwithtenelements,threadBcallsdeleteLastonthesameVector,andtheoperationsareinterleavedasshown inFigure5.1,getLastthrowsArrayIndexOutOfBoundsException.Betweenthecalltosizeandthesubsequentcallto getingetLast,theVectorshrankandtheindexcomputedinthefirststepisnolongervalid.Thisisperfectlyconsistent withthespecificationofVectoritthrowsanexceptionifaskedforanonexistentelement.Butthisisnotwhatacaller expects getLast to do, even in the face of concurrent modification, unless perhaps the Vector was empty to begin with.
Figure5.1.InterleavingofGetlastandDeletelastthatthrowsArrayIndexOutOfBoundsException.
52 JavaConcurrencyInPractice
Because the synchronized collections commit to a synchronization policy that supports clientͲside locking, [1] it is possibletocreatenewoperationsthatareatomicwithrespecttoothercollectionoperationsaslongasweknowwhich locktouse.Thesynchronizedcollectionclassesguardeachmethodwiththelockonthesynchronizedcollectionobject itself. By acquiring the collection lock we can make getLast and deleteLast atomic, ensuring that the size of the Vectordoesnotchangebetweencallingsizeandget,asshowninListing5.2.
[1]ThisisdocumentedonlyobliquelyintheJava5.0Javadoc,asanexampleofthecorrectiterationidiom.
Theriskthatthesizeofthelistmightchangebetweenacalltosizeandthecorrespondingcalltogetisalsopresent whenweiteratethroughtheelementsofaVectorasshowninListing5.3.
ThisiterationidiomreliesonaleapoffaiththatotherthreadswillnotmodifytheVectorbetweenthecallstosizeand get. In a singleͲthreaded environment, this assumption is perfectly valid, but when other threads may concurrently modify the Vector it can lead to trouble. Just as with getLast, if another thread deletes an element while you are iterating through the Vector and the operations are interleaved unluckily, this iteration idiom throws ArrayIndexOutOfBoundsException.
Listing5.2.CompoundActionsonVectorUsingClientǦsideLocking.
public static Object getLast(Vector list) {
synchronized (list) {
int lastIndex = list.size() - 1;
return list.get(lastIndex);
}
}
public static void deleteLast(Vector list) {
synchronized (list) {
int lastIndex = list.size() - 1;
list.remove(lastIndex);
}
}
Listing5.3.IterationthatmayThrowArrayIndexOutOfBoundsException.
for (int i = 0; i < vector.size(); i++)
doSomething(vector.get(i));
EventhoughtheiterationinListing5.3canthrowanexception,thisdoesn'tmeanVectorisn'tthreadͲsafe.Thestateof theVectorisstillvalidandtheexceptionisinfactinconformancewithitsspecification.However,thatsomethingas mundaneasfetchingthelastelementoriterationthrowanexceptionisclearlyundesirable.
TheproblemofunreliableiterationcanagainbeaddressedbyclientͲsidelocking,atsomeadditionalcosttoscalability.
By holding the Vector lock for the duration of iteration, as shown in Listing 5.4, we prevent other threads from modifying the Vector while we are iterating it. Unfortunately, we also prevent other threads from accessing it at all duringthistime,impairingconcurrency.
Listing5.4.IterationwithClientǦsideLocking.
synchronized (vector) {
for (int i = 0; i < vector.size(); i++)
doSomething(vector.get(i));
}
5.1.2.IteratorsandConcurrentmodificationexception
WeuseVectorforthesakeofclarityinmanyofourexamples,eventhoughitisconsidereda"legacy"collectionclass.
But the more "modern" collection classes do not eliminate the problem of compound actions. The standard way to iterateaCollectioniswithanIterator,eitherexplicitlyorthroughtheforͲeachloopsyntaxintroducedinJava5.0, butusingiteratorsdoesnotobviatetheneedtolockthecollectionduringiterationifotherthreadscanconcurrently modifyit.Theiteratorsreturnedbythesynchronizedcollectionsarenotdesignedtodealwithconcurrentmodification, andtheyarefailͲfastͲmeaningthatiftheydetectthatthecollectionhaschangedsinceiterationbegan,theythrowthe uncheckedConcurrentModificationException.
ThesefailͲfastiteratorsarenotdesignedtobefoolproofͲtheyaredesignedtocatchconcurrencyerrorsona"goodͲ
faithͲeffort" basis and thus act only as earlyͲwarning indicators for concurrency problems. They are implemented by associatingamodificationcountwiththecollection:ifthemodificationcountchangesduringiteration,hasNextornext throwsConcurrentModificationException.However,thischeckisdonewithoutsynchronization,sothereisariskof seeingastalevalueofthemodificationcountandthereforethattheiteratordoesnotrealizeamodificationhasbeen made.Thiswasadeliberatedesigntradeofftoreducetheperformanceimpactoftheconcurrentmodificationdetection code.[2]
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
53
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
[2]ConcurrentModificationExceptioncanariseinsingleͲthreadedcodeaswell;thishappenswhenobjectsareremovedfromthecollectiondirectly ratherthanthroughIterator.remove.
Listing5.5illustratesiteratingacollectionwiththeforͲeachloopsyntax.Internally,javacgeneratescodethatusesan Iterator,repeatedlycallinghasNextandnexttoiteratetheList.JustaswithiteratingtheVector,thewaytoprevent ConcurrentModificationExceptionistoholdthecollectionlockforthedurationoftheiteration.
Listing5.5.IteratingaListwithanIterator.
List<Widget> widgetList
= Collections.synchronizedList(new ArrayList<Widget>());
...
// May throw ConcurrentModificationException
for (Widget w : widgetList)
doSomething(w);
Thereareseveralreasons,however,whylockinga collectionduringiterationmaybeundesirable.Otherthreadsthat needtoaccessthecollectionwillblockuntiltheiterationiscomplete;ifthecollectionislargeorthetaskperformedfor eachelementislengthy,theycouldwaitalongtime.Also,ifthecollectionis lockedasinListing5.4, doSomethingis beingcalledwithalockheld,whichisariskfactorfordeadlock(seeChapter10).Evenintheabsenceofstarvationor deadlockrisk,lockingcollectionsforsignificantperiodsoftimehurtsapplicationscalability.Thelongeralockisheld,the morelikelyitistobecontended,andifmanythreadsareblockedwaitingforalockthroughputandCPUutilizationcan suffer(seeChapter11).
Analternativetolockingthecollectionduringiterationistoclonethecollectionanditeratethecopyinstead.Sincethe clone is threadͲconfined, no other thread can modify it during iteration, eliminating the possibility of ConcurrentModificationException.(Thecollectionstillmustbelockedduringthecloneoperationitself.)Cloningthe collectionhasanobviousperformancecost;whetherthisisafavorabletradeoffdependsonmanyfactorsincludingthe sizeofthecollection,howmuchworkisdoneforeachelement,therelativefrequencyofiterationcomparedtoother collectionoperations,andresponsivenessandthroughputrequirements.
5.1.3.HiddenIterators
WhilelockingcanpreventiteratorsfromthrowingConcurrentModificationException,youhavetoremembertouse locking everywhere a shared collection might be iterated. This is trickier than it sounds, as iterators are sometimes hidden, as in HiddenIterator in Listing 5.6. There is no explicit iteration in HiddenIterator, but the code in bold entails iteration just the same. The string concatenation gets turned by the compiler into a call to StringBuilder.append(Object),whichinturninvokesthecollection'stoStringmethodͲandtheimplementationof toString in the standard collections iterates the collection and calls toString on each element to produce a nicely formattedrepresentationofthecollection'scontents.
TheaddTenThingsmethodcouldthrowConcurrentModificationException,becausethecollectionisbeingiterated bytoStringintheprocessofpreparingthedebuggingmessage.Ofcourse,therealproblemisthatHiddenIteratoris notthreadͲsafe;theHiddenIteratorlockshouldbeacquiredbeforeusingsetintheprintlncall,butdebuggingand loggingcodecommonlyneglecttodothis.
Thereallessonhereisthatthegreaterthedistancebetweenthestateandthesynchronizationthatguardsit,themore likely that someone will forget to use proper synchronization when accessing that state. If HiddenIterator wrapped theHashSetwithasynchronizedSet,encapsulatingthesynchronization,thissortoferrorwouldnotoccur.
Justasencapsulatinganobject'sstatemakesiteasiertopreserveitsinvariants,encapsulatingitssynchronizationmakes iteasiertoenforceitssynchronizationpolicy.
54 JavaConcurrencyInPractice
Listing5.6.IterationHiddenwithinStringConcatenation.
public class HiddenIterator {
@GuardedBy("this")
private final Set<Integer> set = new HashSet<Integer>(); public synchronized void add(Integer i) { set.add(i); }
public synchronized void remove(Integer i) { set.remove(i); }
public void addTenThings() {
Random r = new Random();
for (int i = 0; i < 10; i++)
add(r.nextInt());
System.out.println("DEBUG: added ten elements to " + set);
}
}
Iteration is also indirectly invoked by the collection's hashCode and equals methods, which may be called if the collection is used as an element or key of another collection. Similarly, the containsAll, removeAll, and retainAll methods,aswellastheconstructorsthattakecollectionsarearguments,alsoiteratethecollection.Alloftheseindirect usesofiterationcancauseConcurrentModificationException.
5.2.ConcurrentCollections
Java 5.0 improves on the synchronized collections by providing several concurrent collection classes. Synchronized collectionsachievetheirthreadsafetybyserializingallaccesstothecollection'sstate.Thecostofthisapproachispoor concurrency;whenmultiplethreadscontendforthecollectionͲwidelock,throughputsuffers.
Theconcurrentcollections,ontheotherhand,aredesignedforconcurrentaccessfrommultiplethreads.Java5.0adds ConcurrentHashMap,areplacementforsynchronizedhashͲbasedMapimplementations,andCopyOnWriteArrayList,a replacement for synchronized List implementations for cases where traversal is the dominant operation. The new ConcurrentMap interface adds support for common compound actions such as putͲifͲabsent, replace, and conditional remove.
Replacing synchronized collections with concurrent collections can offer dramatic scalability improvements with little risk.
Java5.0alsoaddstwonewcollectiontypes,QueueandBlockingQueue.AQueueisintendedtoholdasetofelements temporarilywhiletheyawaitprocessing.Severalimplementationsareprovided,includingConcurrentLinkedQueue,a traditionalFIFOqueue,andPriorityQueue,a(nonconcurrent)priorityorderedqueue.Queueoperationsdonotblock; if the queue is empty, the retrieval operation returns null. While you can simulate the behavior of a Queue with a Listinfact,LinkedListalsoimplementsQueueͲtheQueueclasseswereaddedbecauseeliminatingtherandomͲaccess requirementsofListadmitsmoreefficientconcurrentimplementations.
BlockingQueue extends Queue to add blocking insertion and retrieval operations. If the queue is empty, a retrieval blocksuntilanelementisavailable,andifthequeueisfull(forboundedqueues)aninsertionblocksuntilthereisspace available. Blocking queues are extremely useful in producerͲconsumer designs, and are covered in greater detail in Section5.3.
Just as ConcurrentHashMap is a concurrent replacement for a synchronized hashͲbased Map, Java 6 adds ConcurrentSkipListMap and ConcurrentSkipListSet, which are concurrent replacements for a synchronized SortedMaporSortedSet(suchasTreeMaporTreeSetwrappedwithsynchronizedMap).
5.2.1.ConcurrentHashMap
The synchronized collections classes hold a lock for the duration of each operation. Some operations, such as HashMap.getorList.contains,mayinvolvemoreworkthanisinitiallyobvious:traversingahashbucketorlisttofind aspecificobjectentailscallingequals(whichitselfmayinvolveafairamountofcomputation)onanumberofcandidate objects. In a hashͲbased collection, if hashCode does not spread out hash values well, elements may be unevenly distributed among buckets; in the degenerate case, a poor hash function will turn a hash table into a linked list.
Traversingalonglistandcallingequalsonsomeoralloftheelementscantakealongtime,andduringthattimeno otherthreadcanaccessthecollection.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
55
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
ConcurrentHashMapisahashͲbasedMaplikeHashMap,butitusesanentirelydifferentlockingstrategythatoffersbetter concurrency and scalability. Instead of synchronizing every method on a common lock, restricting access to a single threadatatime,itusesafinerͲgrainedlocking mechanism calledlockstriping(seeSection11.4.3)to allowagreater degreeofsharedaccess.Arbitrarilymanyreadingthreadscanaccessthemapconcurrently,readerscanaccessthemap concurrently with writers, and a limited number of writers can modify the map concurrently. The result is far higher throughputunderconcurrentaccess,withlittleperformancepenaltyforsingleͲthreadedaccess.
ConcurrentHashMap, along with the other concurrent collections, further improve on the synchronized collection classesbyprovidingiteratorsthatdonotthrowConcurrentModificationException,thuseliminatingtheneedtolock thecollectionduringiteration.TheiteratorsreturnedbyConcurrentHashMapareweaklyconsistentinsteadoffailͲfast.
Aweaklyconsistentiteratorcantolerateconcurrentmodification,traverseselementsastheyexistedwhentheiterator wasconstructed,andmay(butisnotguaranteedto)reflectmodificationstothecollectionaftertheconstructionofthe iterator.
Aswithallimprovements,therearestillafewtradeoffs.ThesemanticsofmethodsthatoperateontheentireMap,such assizeandisEmpty,havebeenslightlyweakenedtoreflecttheconcurrentnatureofthecollection.Sincetheresultof size could be out of date by the time it is computed, it is really only an estimate, so size is allowed to return an approximation instead of an exact count. While at first this may seem disturbing, in reality methods like size and isEmpty are far less useful in concurrent environments because these quantities are moving targets. So the requirements for these operations were weakened to enable performance optimizations for the most important operations,primarilyget,put,containsKey,andremove.
TheonefeatureofferedbythesynchronizedMapimplementationsbutnotbyConcurrentHashMapistheabilitytolock themapforexclusiveaccess.WithHashtableandsynchronizedMap,acquiringtheMaplockpreventsanyotherthread fromaccessingit.Thismightbenecessaryinunusualcasessuchasaddingseveralmappingsatomically,oriteratingthe Mapseveraltimesandneedingtoseethesameelementsinthesameorder.Onthewhole,though,thisisareasonable tradeoff:concurrentcollectionsshouldbeexpectedtochangetheircontentscontinuously.
BecauseithassomanyadvantagesandsofewdisadvantagescomparedtoHashtableorsynchronizedMap,replacing synchronizedMapimplementationswithConcurrentHashMapinmostcasesresultsonlyinbetterscalability.Onlyifyour application needs to lock the map for exclusive access [3] is ConcurrentHashMap not an appropriate dropͲin replacement.
[3]OrifyouarerelyingonthesynchronizationsideeffectsofthesynchronizedMapimplementations.
5.2.2.AdditionalAtomicMapOperations
Since a ConcurrentHashMap cannot be locked for exclusive access, we cannot use clientͲside locking to create new atomicoperationssuchasputͲifͲabsent,aswedidforVectorinSection4.4.1.Instead,anumberofcommoncompound operations such as putͲifͲabsent, removeͲifͲequal, and replaceͲifͲequal are implemented as atomic operations and specified by the ConcurrentMap interface, shown in Listing 5.7. If you find yourself adding such functionality to an existingsynchronizedMapimplementation,itisprobablyasignthatyoushouldconsiderusingaConcurrentMapinstead.
5.2.3.CopyOnWriteArrayList
CopyOnWriteArrayList is a concurrent replacement for a synchronized List that offers better concurrency in some common situations and eliminates the need to lock or copy the collection during iteration. (Similarly, CopyOnWriteArraySetisaconcurrentreplacementforasynchronizedSet.)
ThecopyͲonͲwritecollectionsderivetheirthreadsafetyfromthefactthataslongasaneffectivelyimmutableobjectis properlypublished,nofurthersynchronizationisrequiredwhenaccessingit.Theyimplementmutabilitybycreatingand republishinganewcopyofthecollectioneverytimeitismodified.IteratorsforthecopyͲonͲwritecollectionsretaina referencetothebackingarraythatwascurrentatthestartofiteration,andsincethiswillneverchange,theyneedto synchronizeonlybrieflytoensurevisibilityofthearraycontents.Asaresult,multiplethreadscaniteratethecollection withoutinterferencefromoneanotherorfromthreadswantingtomodifythecollection.Theiteratorsreturnedbythe copyͲonͲwrite collections do not throw ConcurrentModificationException and return the elements exactly as they wereatthetimetheiteratorwascreated,regardlessofsubsequentmodifications.
56 JavaConcurrencyInPractice
Listing5.7. ConcurrentMapInterface.
public interface ConcurrentMap<K,V> extends Map<K,V> {
// Insert into map only if no value is mapped from K
V putIfAbsent(K key, V value);
// Remove only if K is mapped to V
boolean remove(K key, V value);
// Replace value only if K is mapped to oldValue
boolean replace(K key, V oldValue, V newValue);
// Replace value only if K is mapped to some value
V replace(K key, V newValue);
}
Obviously, there is some cost to copying the backing array every time the collection is modified, especially if the collection is large; the copyͲonͲwrite collections are reasonable to use only when iteration is far more common than modification.ThiscriterionexactlydescribesmanyeventͲnotificationsystems:deliveringanotificationrequiresiterating the list of registered listeners and calling each one of them, and in most cases registering or unregistering an event listenerisfarlesscommonthanreceivinganeventnotification.(See[CPJ2.4.4]formoreinformationoncopyͲonͲwrite.) 5.3.BlockingQueuesandtheProducerǦconsumerPattern
Blockingqueuesprovideblockingputandtakemethodsaswellasthetimedequivalentsofferandpoll.Ifthequeue isfull,putblocksuntilspacebecomesavailable;ifthequeueisempty,takeblocksuntilanelementisavailable.Queues canbeboundedorunbounded;unboundedqueuesareneverfull,soaputonanunboundedqueueneverblocks.
Blocking queues support the producerͲconsumer design pattern. A producerͲconsumer design separates the identification of work to be done from the execution of that work by placing work items on a "to do" list for later processing,ratherthanprocessingthemimmediatelyastheyareidentified.TheproducerͲconsumerpatternsimplifies developmentbecauseitremovescodedependenciesbetweenproducerandconsumerclasses,andsimplifiesworkload managementbydecouplingactivitiesthatmayproduceorconsumedataatdifferentorvariablerates.
In a producerͲconsumer design built around a blocking queue, producers place data onto the queue as it becomes available,andconsumersretrievedatafromthequeuewhentheyarereadytotaketheappropriateaction.Producers don'tneedtoknowanythingabouttheidentityornumberofconsumers,orevenwhethertheyaretheonlyproducerͲ
all they have to do is place data items on the queue. Similarly, consumers need not know who the producers are or where the work came from. BlockingQueue simplifies the implementation of producerͲconsumer designs with any numberofproducersandconsumers.OneofthemostcommonproducerͲconsumerdesignsisathreadpoolcoupled withaworkqueue;thispatternisembodiedintheExecutortaskexecutionframeworkthatisthesubjectofChapters6
and8.
The familiar division of labor for two people washing the dishes is an example of a producerͲconsumer design: one personwashesthedishesandplacestheminthedishrack,andtheotherpersonretrievesthedishesfromtherackand driesthem.Inthisscenario,thedishrackactsasablockingqueue;iftherearenodishesintherack,theconsumerwaits untiltherearedishestodry,andiftherackfillsup,theproducerhastostopwashinguntilthereismorespace.This analogy extends to multiple producers (though there may be contention for the sink) and multiple consumers; each workerinteractsonlywiththedishrack.Nooneneedstoknowhowmanyproducersorconsumersthereare,orwho producedagivenitemofwork.
The labels "producer" and "consumer" are relative; an activity that acts as a consumer in one context may act as a producer in another. Drying the dishes "consumes" clean wet dishes and "produces" clean dry dishes. A third person wantingtohelpmightputawaythedrydishes,inwhichcasethedrierisbothaconsumerandaproducer,andthereare nowtwosharedworkqueues(eachofwhichmayblockthedrierfromproceeding.) Blocking queues simplify the coding of consumers, since take blocks until data is available. If the producers don't generate work fast enough to keep the consumers busy, the consumers just wait until more work is available.
Sometimesthisisperfectlyacceptable(asinaserverapplicationwhennoclientisrequestingservice),andsometimesit indicatesthattheratioofproducerthreadstoconsumerthreadsshouldbeadjustedtoachievebetterutilization(asina webcrawlerorotherapplicationinwhichthereiseffectivelyinfiniteworktodo).
Iftheproducersconsistentlygenerateworkfasterthantheconsumerscanprocessit,eventuallytheapplicationwillrun outofmemorybecauseworkitemswillqueueupwithoutbound.Again,theblockingnatureofputgreatlysimplifies coding of producers; if we use a bounded queue, then when the queue fills up the producers block, giving the consumerstimetocatchupbecauseablockedproducercannotgeneratemorework.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
57
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
Blocking queues also provide an offer method, which returns a failure status if the item cannot be enqueued. This enables you to create more flexible policies for dealing with overload, such as shedding load, serializing excess work items and writing them to disk, reducing the number of producer threads, or throttling producers in some other manner.
Boundedqueuesareapowerfulresourcemanagementtoolforbuildingreliableapplications:theymakeyourprogram morerobusttooverloadbythrottlingactivitiesthatthreatentoproducemoreworkthancanbehandled.
While the producerͲconsumer pattern enables producer and consumer code to be decoupled from each other, their behavior is still coupled indirectly through the shared work queue. It is tempting to assume that the consumers will always keep up, so that you need not place any bounds on the size of work queues, but this is a prescription for rearchitectingyoursystemlater.BuildresourcemanagementintoyourdesignearlyusingblockingqueuesͲitisalot easier to do this up front than to retrofit it later. Blocking queues make this easy for a number of situations, but if blocking queues don't fit easily into your design, you can create other blocking data structures using Semaphore (see Section5.5.3).
TheclasslibrarycontainsseveralimplementationsofBlockingQueue.LinkedBlockingQueueandArrayBlockingQueue areFIFOqueues,analogoustoLinkedListandArrayListbutwithbetterconcurrentperformancethanasynchronized List. PriorityBlockingQueue is a priorityͲordered queue, which is useful when you want to process elements in an orderotherthanFIFO.Justlikeothersortedcollections,PriorityBlockingQueuecancompareelementsaccordingto theirnaturalorder(iftheyimplementComparable)orusingaComparator.
ThelastBlockingQueueimplementation,SynchronousQueue,isnotreallyaqueueatall,inthatitmaintainsnostorage spaceforqueuedelements.Instead,itmaintainsalistofqueuedthreadswaitingtoenqueueordequeueanelement.In thedishͲwashinganalogy,thiswouldbelikehavingnodishrack,butinsteadhandingthewasheddishesdirectlytothe nextavailabledryer.Whilethismayseemastrangewaytoimplementaqueue,itreducesthelatencyassociatedwith moving data from producer to consumer because the work can be handed off directly. (In a traditional queue, the enqueue and dequeue operations must complete sequentially before a unit of work can be handed off.) The direct handoffalsofeedsbackmoreinformationaboutthestateofthetasktotheproducer;whenthehandoffisaccepted,it knowsaconsumerhastakenresponsibilityforit,ratherthansimplylettingitsitonaqueuesomewhereͲmuchlikethe differencebetweenhandingadocumenttoacolleagueandmerelyputtingitinhermailboxandhopingshegetsitsoon.
SinceaSynchronousQueuehasnostoragecapacity,putandtakewillblockunlessanotherthreadisalreadywaitingto participateinthehandoff.Synchronousqueuesaregenerallysuitableonlywhenthereareenoughconsumersthatthere nearlyalwayswillbeonereadytotakethehandoff.
5.3.1.Example:DesktopSearch
One type of program that is amenable to decomposition into producers and consumers is an agent that scans local drivesfordocumentsandindexesthemforlatersearching,similartoGoogleDesktoportheWindowsIndexingservice.
DiskCrawlerinListing5.8showsaproducertaskthatsearchesafilehierarchyforfilesmeetinganindexing criterion andputstheirnamesontheworkqueue;IndexerinListing5.8showstheconsumertaskthattakesfilenamesfromthe queueandindexesthem.
The producerͲconsumer pattern offers a threadͲfriendly means of decomposing the desktop search problem into simplercomponents.FactoringfileͲcrawlingandindexingintoseparateactivitiesresultsincodethatismorereadable andreusablethanwithamonolithicactivitythatdoesboth;eachoftheactivitieshasonlyasingletasktodo,andthe blockingqueuehandlesalltheflowcontrol,sothecodeforeachissimplerandclearer.
The producerͲconsumer pattern also enables several performance benefits. Producers and consumers can execute concurrently; if one is I/OͲbound and the other is CPUͲbound, executing them concurrently yields better overall throughput than executing them sequentially. If the producer and consumer activities are parallelizable to different degrees,tightlycouplingthemreducesparallelizabilitytothatofthelessparallelizableactivity.
Listing5.9startsseveralcrawlersandindexers,eachintheirownthread.Aswritten,theconsumerthreadsneverexit, whichpreventstheprogramfromterminating;weexamineseveraltechniquesforaddressingthisprobleminChapter7.
While this example uses explicitly managed threads, many producerͲconsumer designs can be expressed using the Executortaskexecutionframework,whichitselfusestheproducerͲconsumerpattern.
5.3.2.SerialThreadConfinement
Theblockingqueueimplementationsinjava.util.concurrentallcontainsufficientinternalsynchronizationtosafely publishobjectsfromaproducerthreadtotheconsumerthread.
58 JavaConcurrencyInPractice
Formutableobjects,producerͲconsumerdesignsandblockingqueuesfacilitateserialthreadconfinementforhanding offownershipofobjectsfromproducerstoconsumers.AthreadͲconfinedobjectisownedexclusivelybyasinglethread, butthatownershipcanbe"transferred"bypublishingitsafelywhereonlyoneotherthreadwillgainaccesstoitand ensuringthatthepublishingthreaddoesnotaccessitafterthehandoff.Thesafepublicationensuresthattheobject's state is visible to the new owner, and since the original owner will not touch it again, it is now confined to the new thread.Thenewownermaymodifyitfreelysinceithasexclusiveaccess.
Objectpoolsexploitserialthreadconfinement,"lending"anobjecttoarequestingthread.Aslongasthepoolcontains sufficient internal synchronization to publish the pooled object safely, and as long as the clients do not themselves publish the pooled object or use it after returning it to the pool, ownership can be transferred safely from thread to thread.
Onecouldalsouseotherpublicationmechanismsfortransferringownershipofamutableobject,butitisnecessaryto ensurethat onlyonethreadreceivestheobjectbeinghandedoff.Blockingqueuesmake thiseasy;withalittlemore work, it could also done with the atomic remove method of ConcurrentMap or the compareAndSet method of AtomicReference.
Listing5.8.ProducerandConsumerTasksinaDesktopSearchApplication.
public class FileCrawler implements Runnable {
private final BlockingQueue<File> fileQueue;
private final FileFilter fileFilter;
private final File root;
...
public void run() {
try {
crawl(root);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private void crawl(File root) throws InterruptedException {
File[] entries = root.listFiles(fileFilter);
if (entries != null) {
for (File entry : entries)
if (entry.isDirectory())
crawl(entry);
else if (!alreadyIndexed(entry))
fileQueue.put(entry);
}
}
}
public class Indexer implements Runnable {
private final BlockingQueue<File> queue;
public Indexer(BlockingQueue<File> queue) {
this.queue = queue;
}
public void run() {
try {
while (true)
indexFile(queue.take());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
Listing5.9.StartingtheDesktopSearch.
public static void startIndexing(File[] roots) {
BlockingQueue<File> queue = new LinkedBlockingQueue<File>(BOUND); FileFilter filter = new FileFilter() {
public boolean accept(File file) { return true; }
};
for (File root : roots)
new Thread(new FileCrawler(queue, filter, root)).start();
for (int i = 0; i < N_CONSUMERS; i++)
new Thread(new Indexer(queue)).start();
}
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
59
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
5.3.3.DequesandWorkStealing
Java6alsoaddsanothertwocollectiontypes,Deque(pronounced"deck")andBlockingDeque,thatextendQueueand BlockingQueue.ADequeisadoubleͲendedqueuethatallowsefficientinsertionandremovalfromboththeheadand thetail.ImplementationsincludeArrayDequeandLinkedBlockingDeque.
Just as blocking queues lend themselves to the producerͲconsumer pattern, deques lend themselves to a related pattern called work stealing. A producerͲconsumer design has one shared work queue for all consumers; in a work stealingdesign,everyconsumerhasitsowndeque.Ifaconsumerexhauststheworkinitsowndeque,itcanstealwork fromthetailofsomeoneelse'sdeque.WorkstealingcanbemorescalablethanatraditionalproducerͲconsumerdesign becauseworkersdon'tcontendforasharedworkqueue;mostofthetimetheyaccessonlytheirowndeque,reducing contention. When a worker has to access another's queue, it does so from the tail rather than the head, further reducingcontention.
Work stealing is well suited to problems in which consumers are also producers Ͳ when performing a unit of work is likelytoresultintheidentificationofmorework.Forexample,processingapageinawebcrawlerusuallyresultsinthe identificationofnewpagestobecrawled.Similarly,manygraphͲexploringalgorithms,suchasmarkingtheheapduring garbagecollection,canbeefficientlyparallelizedusingworkstealing.Whenaworkeridentifiesanewunitofwork,it placesitattheendofitsowndeque(oralternatively,inaworksharingdesign,onthatofanotherworker);whenits dequeisempty,itlooksforworkattheendofsomeoneelse'sdeque,ensuringthateachworkerstaysbusy.
5.4.BlockingandInterruptibleMethods
Threadsmayblock,orpause,forseveralreasons:waitingforI/Ocompletion,waitingtoacquirealock,waitingtowake upfromThread.sleep,orwaitingfortheresultofacomputationinanotherthread.Whenathreadblocks,itisusually suspended and placed in one of the blocked thread states (BLOCKED, WAITING, or TIMED_WAITING). The distinction betweenablockingoperationandanordinaryoperationthatmerelytakesalongtimetofinishisthatablockedthread mustwaitforaneventthatisbeyonditscontrolbeforeitcanproceedͲtheI/Ocompletes,thelockbecomesavailable, ortheexternalcomputationfinishes.Whenthatexternaleventoccurs,thethreadisplacedbackintheRUNNABLEstate andbecomeseligibleagainforscheduling.
The put and take methods of BlockingQueue throw the checked InterruptedException, as do a number of other librarymethodssuchasThread.sleep.WhenamethodcanthrowInterruptedException,itistellingyouthatitisa blockingmethod,andfurtherthatifitisinterrupted,itwillmakeanefforttostopblockingearly.
Thread provides the interrupt method for interrupting a thread and for querying whether a thread has been interrupted.Eachthreadhasabooleanpropertythatrepresentsitsinterruptedstatus;interruptingathreadsetsthis status.
Interruptionisacooperativemechanism.Onethreadcannotforceanothertostopwhatitisdoinganddosomething else;whenthreadAinterruptsthreadB,AismerelyrequestingthatBstopwhatitisdoingwhenitgetstoaconvenient stoppingpointͲifitfeelslikeit.WhilethereisnothingintheAPIorlanguagespecificationthatdemandsanyspecific applicationͲlevel semantics for interruption, the most sensible use for interruption is to cancel an activity. Blocking methodsthatareresponsivetointerruptionmakeiteasiertocancellongͲrunningactivitiesonatimelybasis.
WhenyourcodecallsamethodthatthrowsInterruptedException,thenyourmethodisablockingmethodtoo,and musthaveaplanforrespondingtointerruption.Forlibrarycode,therearebasicallytwochoices: PropagatetheInterruptedException.ThisisoftenthemostsensiblepolicyifyoucangetawaywithitͲjustpropagate the InterruptedExceptiontoyourcaller.This couldinvolvenotcatching InterruptedException,orcatchingitand throwingitagainafterperformingsomebriefactivityͲspecificcleanup.
Restoretheinterrupt.SometimesyoucannotthrowInterruptedException,forinstancewhenyourcodeispartofa Runnable. In these situations, you must catch InterruptedException and restore the interrupted status by calling interrupt on the current thread, so that code higher up the call stack can see that an interrupt was issued, as demonstratedinListing5.10.
Youcangetmuchmoresophisticatedwithinterruption,butthesetwoapproachesshouldworkinthevastmajorityof situations.ButthereisonethingyoushouldnotdowithInterruptedExceptioncatchitanddonothinginresponse.
Thisdeprivescodehigheruponthecallstackoftheopportunitytoactontheinterruption,becausetheevidencethat thethreadwasinterruptedislost.Theonlysituationinwhichitisacceptabletoswallowaninterruptiswhenyouare extending Thread and therefore control all the code higher up on the call stack. Cancellation and interruption are coveredingreaterdetailinChapter7.
60 JavaConcurrencyInPractice
Listing5.10.RestoringtheInterruptedStatussoasNottoSwallowtheInterrupt.
public class TaskRunnable implements Runnable {
BlockingQueue<Task> queue;
...
public void run() {
try {
processTask(queue.take());
} catch (InterruptedException e) {
// restore interrupted status
Thread.currentThread().interrupt();
}
}
}
5.5.Synchronizers
Blockingqueuesareuniqueamongthecollectionsclasses:notonlydotheyactascontainersforobjects,buttheycan alsocoordinatethecontrolflowofproducerandconsumerthreadsbecausetakeandputblockuntilthequeueenters thedesiredstate(notemptyornotfull).
Asynchronizerisanyobjectthatcoordinatesthecontrolflowofthreadsbasedonitsstate.Blockingqueuescanactas synchronizers; other types of synchronizers include semaphores, barriers, and latches. There are a number of synchronizerclassesintheplatformlibrary;ifthesedonotmeetyourneeds,youcanalsocreateyourownusingthe mechanismsdescribedinChapter14.
Allsynchronizerssharecertainstructuralproperties:theyencapsulatestatethatdetermineswhetherthreadsarrivingat thesynchronizershouldbeallowedtopassorforcedtowait,providemethodstomanipulatethatstate,andprovide methodstowaitefficientlyforthesynchronizertoenterthedesiredstate.
5.5.1.Latches
Alatchisasynchronizerthatcandelaytheprogressofthreadsuntilitreachesitsterminalstate[CPJ3.4.2].Alatchacts asagate:untilthelatchreachestheterminalstatethegateisclosedandnothreadcanpass,andintheterminalstate thegateopens,allowingallthreadstopass.Oncethelatchreachestheterminalstate,itcannotchangestateagain,soit remains open forever. Latches can be used to ensure that certain activities do not proceed until other oneͲtime activitiescomplete,suchas:
x Ensuringthatacomputationdoesnotproceeduntilresourcesitneedshavebeeninitialized.Asimplebinary (twoͲstate) latch could be used to indicate "Resource R has been initialized", and any activity that requires R
wouldwaitfirstonthislatch.
x Ensuringthataservicedoesnotstartuntilotherservicesonwhichitdependshavestarted.Eachservicewould haveanassociatedbinarylatch;startingserviceSwouldinvolvefirstwaitingonthelatchesforotherserviceson whichSdepends,andthenreleasingtheSlatchafterstartupcompletessoanyservicesthatdependonScan thenproceed.
x Waitinguntilallthepartiesinvolvedinanactivity,forinstancetheplayersinamultiͲplayergame,arereadyto proceed.Inthiscase,thelatchreachestheterminalstateafteralltheplayersareready.
CountDownLatch is a flexible latch implementation that can be used in any of these situations; it allows one or more threads to wait for a set of events to occur. The latch state consists of a counter initialized to a positive number, representingthenumberofeventstowaitfor.ThecountDownmethoddecrementsthecounter,indicatingthatanevent has occurred, and the await methods wait for the counter to reach zero, which happens when all the events have occurred. If the counter is nonzero on entry, await blocks until the counter reaches zero, the waiting thread is interrupted,orthewaittimesout.
TestHarnessinListing5.11illustratestwocommonusesforlatches.TestHarnesscreatesanumberofthreadsthatrun agiventaskconcurrently.Itusestwolatches,a"startinggate"andan"endinggate".Thestartinggateisinitializedwith acountofone;theendinggateisinitializedwithacountequaltothenumberofworkerthreads.Thefirstthingeach workerthreaddoesiswaitonthestartinggate;thisensuresthatnoneofthemstartsworkinguntiltheyallarereadyto start.Thelastthingeachdoesiscountdownontheendinggate;thisallowsthemasterthreadtowaitefficientlyuntil thelastoftheworkerthreadshasfinished,soitcancalculatetheelapsedtime.
Why did we bother with the latches in TestHarness instead of just starting the threads immediately after they are created?Presumably,wewantedtomeasurehowlongittakestorunataskntimesconcurrently.Ifwesimplycreated andstartedthethreads,thethreadsstartedearlierwouldhavea"headstart"onthelaterthreads,andthedegreeof contentionwouldvaryovertimeasthenumberofactivethreadsincreasedordecreased.Usingastartinggateallows
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
61
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
themasterthreadtoreleasealltheworkerthreadsatonce,andtheendinggateallowsthemasterthreadtowaitfor thelastthreadtofinishratherthanwaitingsequentiallyforeachthreadtofinish.
5.5.2.FutureTask
FutureTask also acts like a latch. (FutureTask implements Future, which describes an abstract resultͲbearing computation [CPJ 4.3.3].) A computation represented by a FutureTask is implemented with a Callable, the resultͲ
bearingequivalentofRunnable,andcanbeinoneofthreestates:waitingtorun,running,orcompleted.Completion subsumesallthewaysacomputationcancomplete,includingnormalcompletion,cancellation,andexception.Oncea FutureTaskentersthecompletedstate,itstaysinthatstateforever.
ThebehaviorofFuture.getdependsonthestateofthetask.Ifitiscompleted,getreturnstheresultimmediately,and otherwiseblocksuntilthetasktransitionstothecompletedstateandthenreturnstheresultorthrowsanexception.
FutureTask conveys the result from the thread executing the computation to the thread(s) retrieving the result; the specificationofFutureTaskguaranteesthatthistransferconstitutesasafepublicationoftheresult.
Listing5.11.UsingCountDownLatchforStartingandStoppingThreadsinTimingTests.
public class TestHarness {
public long timeTasks(int nThreads, final Runnable task)
throws InterruptedException {
final CountDownLatch startGate = new CountDownLatch(1);
final CountDownLatch endGate = new CountDownLatch(nThreads);
for (int i = 0; i < nThreads; i++) {
Thread t = new Thread() {
public void run() {
try {
startGate.await();
try {
task.run();
} finally {
endGate.countDown();
}
} catch (InterruptedException ignored) { }
}
};
t.start();
}
long start = System.nanoTime();
startGate.countDown();
endGate.await();
long end = System.nanoTime();
return end-start;
}
}
FutureTaskisusedbytheExecutorframeworktorepresentasynchronoustasks,andcanalsobeusedtorepresentany potentially lengthy computation that can be started before the results are needed. Preloader in Listing 5.12 uses FutureTasktoperformanexpensivecomputation whoseresultsareneededlater;bystarting the computationearly, youreducethetimeyouwouldhavetowaitlaterwhenyouactuallyneedtheresults.
62 JavaConcurrencyInPractice
Listing5.12.UsingFutureTasktoPreloadDatathatisNeededLater.
public class Preloader {
private final FutureTask<ProductInfo> future =
new FutureTask<ProductInfo>(new Callable<ProductInfo>() {
public ProductInfo call() throws DataLoadException {
return loadProductInfo();
}
});
private final Thread thread = new Thread(future);
public void start() { thread.start(); }
public ProductInfo get()
throws DataLoadException, InterruptedException {
try {
return future.get();
} catch (ExecutionException e) {
Throwable cause = e.getCause();
if (cause instanceof DataLoadException)
throw (DataLoadException) cause;
else
throw launderThrowable(cause);
}
}
}
PreloadercreatesaFutureTaskthatdescribesthetaskofloadingproductinformationfromadatabaseandathreadin whichthecomputationwillbeperformed.Itprovidesastartmethodtostartthethread,sinceitisinadvisabletostart athreadfromaconstructororstaticinitializer.WhentheprogramlaterneedstheProductInfo,itcancallget,which returnstheloadeddataifitisready,orwaitsfortheloadtocompleteifnot.
Tasks described by Callable can throw checked and unchecked exceptions, and any code can throw an Error.
Whatever the task code may throw, it is wrapped in an ExecutionException and rethrown from Future.get. This complicates code that calls get, not only because it must deal with the possibility of ExecutionException (and the unchecked CancellationException), but also because the cause of the ExecutionException is returned as a THRowable,whichisinconvenienttodealwith.
When get throws an ExecutionException in Preloader, the cause will fall into one of three categories: a checked exceptionthrownbytheCallable,aRuntimeException,oranError.Wemusthandleeachofthesecasesseparately, but we will use the launderThrowable utility method in Listing 5.13 to encapsulate some of the messier exceptionͲ
handling logic. Before calling launderThrowable, Preloader tests for the known checked exceptions and rethrows them.Thatleavesonlyuncheckedexceptions,whichPreloaderhandlesbycallinglaunderThrowableandthrowingthe result.IftheThrowablepassedtolaunderThrowableisanError,launderThrowablerethrowsitdirectly;ifitisnota RuntimeException,itthrowsanIllegalStateExceptiontoindicatealogicerror.ThatleavesonlyRuntimeException, whichlaunderThrowablereturnstoitscaller,andwhichthecallergenerallyrethrows.
Listing5.13.CoercinganUncheckedThrowabletoaRuntimeException.
/** If the Throwable is an Error, throw it; if it is a
* RuntimeException return it, otherwise throw IllegalStateException
*/
public static RuntimeException launderThrowable(Throwable t) {
if (t instanceof RuntimeException)
return (RuntimeException) t;
else if (t instanceof Error)
throw (Error) t;
else
throw new IllegalStateException("Not unchecked", t);
}
5.5.3.Semaphores
Countingsemaphoresareusedtocontrolthenumberofactivitiesthatcanaccessacertainresourceorperformagiven action at the same time [CPJ 3.4.1]. Counting semaphores can be used to implement resource pools or to impose a boundonacollection.
A Semaphore manages a set of virtual permits; the initial number of permits is passed to the Semaphore constructor.
Activitiescanacquirepermits(aslongassomeremain)andreleasepermitswhentheyaredonewiththem.Ifnopermit isavailable,acquireblocksuntiloneis(oruntilinterruptedortheoperationtimesout).Thereleasemethodreturnsa permittothesemaphore.[4]Adegeneratecaseofacountingsemaphoreisabinarysemaphore,aSemaphorewithan initialcountofone.AbinarysemaphorecanbeusedasamutexwithnonͲreentrantlockingsemantics;whoeverholds thesolepermitholdsthemutex.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
63
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
[4]Theimplementationhasnoactualpermitobjects,andSemaphoredoesnotassociatedispensedpermitswiththreads,soapermitacquiredin onethreadcanbereleasedfromanotherthread.Youcanthinkofacquireasconsumingapermitandreleaseascreatingone;aSemaphoreisnot limitedtothenumberofpermitsitwascreatedwith.
Semaphoresareusefulforimplementingresourcepoolssuchasdatabaseconnectionpools.Whileitiseasytoconstruct afixedͲsizedpoolthatfailsifyourequestaresourcefromanemptypool,whatyoureallywantistoblockifthepoolis emptyandunblockwhenitbecomesnonemptyagain.IfyouinitializeaSemaphoretothepoolsize,acquireapermit before trying to fetch a resource from the pool, and release the permit after putting a resource back in the pool, acquireblocksuntilthepoolbecomesnonempty.ThistechniqueisusedintheboundedbufferclassinChapter12.(An easierwaytoconstructablockingobjectpoolwouldbetouseaBlockingQueuetoholdthepooledresources.) Similarly, you can use a Semaphore to turn any collection into a blocking bounded collection, as illustrated by BoundedHashSet in Listing 5.14. The semaphore is initialized to the desired maximum size of the collection. The add operationacquiresapermitbeforeaddingtheitemintotheunderlyingcollection.Iftheunderlyingaddoperationdoes notactuallyaddanything,itreleasesthepermitimmediately.Similarly,asuccessfulremoveoperationreleasesapermit, enabling more elements to be added. The underlying Set implementation knows nothing about the bound; this is handledbyBoundedHashSet.
5.5.4.Barriers
Wehaveseenhowlatchescanfacilitatestartingagroupofrelatedactivitiesorwaitingforagroupofrelatedactivities tocomplete.LatchesaresingleͲuseobjects;oncealatchenterstheterminalstate,itcannotbereset.
Barriersaresimilartolatchesinthattheyblockagroupofthreadsuntilsomeeventhasoccurred[CPJ4.4.3].Thekey difference is that with a barrier, all the threads must come together at a barrier point at the same time in order to proceed.Latchesareforwaitingforevents;barriersareforwaitingforotherthreads.Abarrierimplementstheprotocol somefamiliesusetorendezvousduringadayatthemall:"EveryonemeetatMcDonald'sat6:00;onceyougetthere, staythereuntileveryoneshowsup,andthenwe'llfigureoutwhatwe'redoingnext."
CyclicBarrier allows a fixed number of parties to rendezvous repeatedly at a barrier point and is useful in parallel iterativealgorithmsthatbreakdownaproblemintoafixednumberofindependentsubproblems.Threadscallawait whentheyreachthebarrierpoint,andawaitblocksuntilallthethreadshavereachedthebarrierpoint.Ifallthreads meetatthebarrierpoint,thebarrierhasbeensuccessfullypassed,inwhichcaseallthreadsarereleasedandthebarrier isresetsoitcanbeusedagain.Ifacalltoawaittimesoutorathreadblockedinawaitisinterrupted,thenthebarrieris considered broken and all outstanding calls to await terminate with BrokenBarrierException. If the barrier is successfully passed, await returns a unique arrival index for each thread, which can be used to "elect" a leader that takessomespecialactioninthenextiteration.CyclicBarrieralsoletsyoupassabarrieractiontotheconstructor; thisisaRunnablethatisexecuted(inoneofthesubtaskthreads)whenthebarrierissuccessfullypassedbutbeforethe blockedthreadsarereleased.
64 JavaConcurrencyInPractice
Listing5.14.UsingSemaphoretoBoundaCollection.
public class BoundedHashSet<T> {
private final Set<T> set;
private final Semaphore sem;
public BoundedHashSet(int bound) {
this.set = Collections.synchronizedSet(new HashSet<T>());
sem = new Semaphore(bound);
}
public boolean add(T o) throws InterruptedException {
sem.acquire();
boolean wasAdded = false;
try {
wasAdded = set.add(o);
return wasAdded;
}
finally {
if (!wasAdded)
sem.release();
}
}
public boolean remove(Object o) {
boolean wasRemoved = set.remove(o);
if (wasRemoved)
sem.release();
return wasRemoved;
}
}
Barriers areoften used in simulations, where the work to calculate one step can be done in parallel but all the work associated with a given step must complete before advancing to the next step. For example, in nͲbody particle simulations,eachstepcalculatesanupdatetothepositionofeachparticlebasedonthelocationsandotherattributes of the other particles. Waiting on a barrier between each update ensures that all updates forstep k have completed beforemovingontostepk+1.
CellularAutomata in Listing 5.15 demonstrates using a barrier to compute a cellular automata simulation, such as Conway's Life game (Gardner, 1970). When parallelizing a simulation, it is generally impractical to assign a separate thread to each element (in the case of Life, a cell); this would require too many threads, and the overhead of coordinating them would dwarf the computation. Instead, it makes sense to partition the problem into a number of subparts,leteachthreadsolveasubpart,andthenmergetheresults.CellularAutomatapartitionstheboardintoNcpu parts,whereNcpuisthenumberofCPUsavailable,andassignseachparttoathread.[5]Ateachstep,theworkerthreads calculatenewvaluesforallthecellsintheirpartoftheboard.Whenallworkerthreadshavereachedthebarrier,the barrieractioncommitsthenewvaluestothedatamodel.Afterthebarrieractionruns,theworkerthreadsarereleased tocomputethenextstepofthecalculation,whichincludesconsultinganisDonemethodtodeterminewhetherfurther iterationsarerequired.
[5]ForcomputationalproblemslikethisthatdonoI/Oandaccessnoshareddata,NcpuorNcpu+1threadsyieldoptimalthroughput;morethreads donothelp,andmayinfactdegradeperformanceasthethreadscompeteforCPUandmemoryresources.
AnotherformofbarrierisExchanger,atwoͲpartybarrierinwhichthepartiesexchangedataatthebarrierpoint[CPJ
3.4.3].Exchangersareusefulwhenthepartiesperformasymmetricactivities,forexamplewhenonethreadfillsabuffer withdataandtheotherthreadconsumesthedatafromthebuffer;thesethreadscoulduseanExchangertomeetand exchange a full buffer for an empty one. When two threads exchange objects via an Exchanger, the exchange constitutesasafepublicationofbothobjectstotheotherparty.
Thetimingoftheexchangedependsontheresponsivenessrequirementsoftheapplication.Thesimplestapproachis thatthefillingtaskexchangeswhenthebufferisfull,andtheemptyingtaskexchangeswhenthebufferisempty;this minimizes the number of exchanges but can delay processing of some data if the arrival rate of new data is unpredictable.Anotherapproachwouldbethatthefillerexchangeswhenthebufferisfull,butalsowhenthebufferis partiallyfilledandacertainamountoftimehaselapsed.
5.6.BuildinganEfficient,ScalableResultCache
Nearlyeveryserverapplicationusessomeformofcaching.Reusingtheresultsofapreviouscomputationcanreduce latencyandincreasethroughput,atthecostofsomeadditionalmemoryusage.
Listing5.15.CoordinatingComputationinaCellularAutomatonwithCyclicBarrier.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
65
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
public class CellularAutomata {
private final Board mainBoard;
private final CyclicBarrier barrier;
private final Worker[] workers;
public CellularAutomata(Board board) {
this.mainBoard = board;
int count = Runtime.getRuntime().availableProcessors();
this.barrier = new CyclicBarrier(count,
new Runnable() {
public void run() {
mainBoard.commitNewValues();
}});
this.workers = new Worker[count];
for (int i = 0; i < count; i++)
workers[i] = new Worker(mainBoard.getSubBoard(count, i));
}
private class Worker implements Runnable {
private final Board board;
public Worker(Board board) { this.board = board; }
public void run() {
while (!board.hasConverged()) {
for (int x = 0; x < board.getMaxX(); x++)
for (int y = 0; y < board.getMaxY(); y++)
board.setNewValue(x, y, computeValue(x, y));
try {
barrier.await();
} catch (InterruptedException ex) {
return;
} catch (BrokenBarrierException ex) {
return;
}
}
}
}
public void start() {
for (int i = 0; i < workers.length; i++)
new Thread(workers[i]).start();
mainBoard.waitForConvergence();}
}
}
Likemanyotherfrequentlyreinventedwheels,cachingoftenlookssimplerthanitis.Anaivecacheimplementationis likely to turn a performance bottleneck into a scalability bottleneck, even if it does improve singleͲthreaded performance.Inthissectionwedevelopanefficientandscalableresultcacheforacomputationallyexpensivefunction.
Let'sstartwith theobviousapproach ͲasimpleHashMapandthenlookatsomeofitsconcurrencydisadvantagesand howtofixthem.
The Computable<A,V> interface in Listing 5.16 describes a function with input of type A and result of type V.
ExpensiveFunction, which implements Computable, takes a long time to compute its result; we'd like to create a Computablewrapperthatrememberstheresultsofpreviouscomputationsandencapsulatesthecachingprocess.(This techniqueisknownasMemorization.)
66 JavaConcurrencyInPractice
Listing5.16.InitialCacheAttemptUsingHashMapandSynchronization.
public interface Computable<A, V> {
V compute(A arg) throws InterruptedException;
}
public class ExpensiveFunction
implements Computable<String, BigInteger> {
public BigInteger compute(String arg) {
// after deep thought...
return new BigInteger(arg);
}
}
public class Memorizer1<A, V> implements Computable<A, V> {
@GuardedBy("this")
private final Map<A, V> cache = new HashMap<A, V>();
private final Computable<A, V> c;
public Memorizer1(Computable<A, V> c) {
this.c = c;
}
public synchronized V compute(A arg) throws InterruptedException {
V result = cache.get(arg);
if (result == null) {
result = c.compute(arg);
cache.put(arg, result);
}
return result;
}
}
Memorizer1 in Listing 5.16 shows a first attempt: using a HashMap to store the results of previous computations. The computemethodfirstcheckswhetherthedesiredresultisalreadycached,andreturnsthepreͲcomputedvalueifitis.
Otherwise,theresultiscomputedandcachedintheHashMapbeforereturning.
HashMap is not threadͲsafe, so to ensure that two threads do not access the HashMap at the same time, Memorizer1
takes the conservative approach of synchronizing the entire compute method. This ensures thread safety but has an obviousscalabilityproblem:onlyonethreadatatimecanexecutecomputeatall.Ifanotherthreadisbusycomputinga result, other threads calling compute may be blocked for a long time. If multiple threads are queued up waiting to compute values not already computed, compute may actually take longer than it would have without Memorization.
Figure5.2illustrateswhatcouldhappenwhenseveralthreadsattempttouseafunctionmemorizedwiththisapproach.
Thisisnotthesortofperformanceimprovementwehadhopedtoachievethroughcaching.
Figure5.2.PoorConcurrencyofMemorizer1.
Memorizer2inListing5.17improvesontheawfulconcurrentbehaviorofMemorizer1byreplacingtheHashMapwitha ConcurrentHashMap. Since ConcurrentHashMap is threadͲsafe, there is no need to synchronize when accessing the backingMap,thuseliminatingtheserializationinducedbysynchronizingcomputeinMemorizer1.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
67
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
Memorizer2 certainly has better concurrent behavior than Memorizer1: multiple threads can actually use it concurrently.ButitstillhassomedefectsasacacheͲthereisawindowofvulnerabilityinwhichtwo threadscalling compute at the same time could end up computing the same value. In the case of memorization, this is merely inefficient Ͳ the purpose of a cache is to prevent the same data from being calculated multiple times. For a more generalͲpurposecachingmechanism,itisfarworse;foranobjectcachethatissupposedtoprovideonceͲandͲonlyͲonce initialization,thisvulnerabilitywouldalsoposeasafetyrisk.
TheproblemwithMemorizer2isthatifonethreadstartsanexpensivecomputation,otherthreadsarenotawarethat the computation is in progress and so may start the same computation, as illustrated in Figure 5.3. We'd like to somehowrepresentthenotionthat"threadXiscurrentlycomputingf(27)",sothatifanotherthreadarriveslookingfor f (27), it knows that the most efficient way to find it is to head over to Thread X's house, hang out there until X is finished,andthenask"Hey,whatdidyougetforf(27)?"
Figure5.3.TwoThreadsComputingtheSameValueWhenUsingMemorizer2.
Listing5.17.ReplacingHashMapwithConcurrentHashMap.
public class Memorizer2<A, V> implements Computable<A, V> {
private final Map<A, V> cache = new ConcurrentHashMap<A, V>(); private final Computable<A, V> c;
public Memorizer2(Computable<A, V> c) { this.c = c; }
public V compute(A arg) throws InterruptedException {
V result = cache.get(arg);
if (result == null) {
result = c.compute(arg);
cache.put(arg, result);
}
return result;
}
}
We'vealreadyseenaclassthatdoesalmostexactlythis:FutureTask.FutureTaskrepresentsacomputationalprocess thatmayormaynotalreadyhavecompleted.FutureTask.getreturnstheresultofthecomputationimmediatelyifitis available;otherwiseitblocksuntiltheresulthasbeencomputedandthenreturnsit.
Memorizer3 in Listing 5.18 redefines the backing Map for the value cache as a ConcurrentHashMap<A,Future<V>> insteadofaConcurrentHashMap<A,V>.Memorizer3firstcheckstoseeiftheappropriatecalculationhasbeenstarted (as opposed to finished, as in Memorizer2). If not, it creates a FutureTask, registers it in the Map, and starts the computation;otherwiseitwaitsfortheresultoftheexistingcomputation.Theresultmightbeavailableimmediatelyor mightbeintheprocessofbeingcomputedͲbutthisistransparenttothecallerofFuture.get.
TheMemorizer3implementationisalmostperfect:itexhibitsverygoodconcurrency(mostlyderivedfromtheexcellent concurrencyofConcurrentHashMap),theresultisreturnedefficientlyifitisalreadyknown,andifthecomputationisin progressbyanotherthread,newlyarrivingthreadswaitpatientlyfortheresult.IthasonlyonedefectͲthereisstilla smallwindowofvulnerabilityinwhichtwothreadsmightcomputethesamevalue.Thiswindowisfarsmallerthanin Memorizer2, but because the if block in compute is still a nonͲatomic checkͲthenͲact sequence, it is possible for two threadstocallcomputewiththesamevalueatroughlythesametime,bothseethatthecachedoesnotcontain the desiredvalue,andbothstartthecomputation.ThisunluckytimingisillustratedinFigure5.4.
68 JavaConcurrencyInPractice
Figure5.4.UnluckyTimingthatcouldCauseMemorizer3toCalculatetheSameValueTwice.
Listing5.18.MemorizingWrapperUsingFutureTask.
public class Memorizer3<A, V> implements Computable<A, V> {
private final Map<A, Future<V>> cache
= new ConcurrentHashMap<A, Future<V>>();
private final Computable<A, V> c;
public Memorizer3(Computable<A, V> c) { this.c = c; }
public V compute(final A arg) throws InterruptedException {
Future<V> f = cache.get(arg);
if (f == null) {
Callable<V> eval = new Callable<V>() {
public V call() throws InterruptedException {
return c.compute(arg);
}
};
FutureTask<V> ft = new FutureTask<V>(eval);
f = ft;
cache.put(arg, ft);
ft.run(); // call to c.compute happens here
}
try {
return f.get();
} catch (ExecutionException e) {
throw launderThrowable(e.getCause());
}
}
}
Memorizer3isvulnerabletothisproblembecauseacompoundaction(putͲifͲabsent)isperformedonthebackingmap that cannot be made atomic using locking. Memorizer in Listing 5.19 takes advantage of the atomic putIfAbsent methodofConcurrentMap,closingthewindowofvulnerabilityinMemorizer3.
Caching a Future instead of a value creates the possibility of cache pollution: if a computation is cancelled or fails, futureattemptstocomputetheresultwillalsoindicatecancellationorfailure.Toavoidthis,Memorizerremovesthe Futurefromthecacheifitdetectsthatthecomputationwascancelled;itmightalsobedesirabletoremovetheFuture upondetectingaRuntimeExceptionifthecomputationmightsucceedonafutureattempt.Memorizeralsodoesnot address cache expiration, but this could be accomplished by using a subclass of FutureTask that associates an expirationtimewitheachresultandperiodicallyscanningthecacheforexpiredentries.(Similarly,itdoesnotaddress cache eviction, where old entries are removed to make room for new ones so that the cache does not consume too muchmemory.)
With our concurrent cache implementation complete, we can now add real caching to the factorizing servlet from Chapter2,aspromised.FactorizerinListing5.20usesMemorizertocachepreviouslycomputedvaluesefficientlyand scalably.
5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45
66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454
69
B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks
Listing5.19.FinalImplementationofMemorizer.
public class Memorizer<A, V> implements Computable<A, V> {
private final ConcurrentMap<A, Future<V>> cache
= new ConcurrentHashMap<A, Future<V>>();
private final Computable<A, V> c;
public Memorizer(Computable<A, V> c) { this.c = c; }
public V compute(final A arg) throws InterruptedException {
while (true) {
Future<V> f = cache.get(arg);
if (f == null) {
Callable<V> eval = new Callable<V>() {
public V call() throws InterruptedException {
return c.compute(arg);
}
};
FutureTask<V> ft = new FutureTask<V>(eval);
f = cache.putIfAbsent(arg, ft);
if (f == null) { f = ft; ft.run(); }
}
try {
return f.get();
} catch (CancellationException e) {
cache.remove(arg, f);
} catch (ExecutionException e) {
throw launderThrowable(e.getCause());
}
}
}
}
Listing5.20.FactorizingServletthatCachesResultsUsingMemorizer.
@ThreadSafe
public class Factorizer implements Servlet {
private final Computable<BigInteger, BigInteger[]> c =
new Computable<BigInteger, BigInteger[]>() {
public BigInteger[] compute(BigInteger arg) {
return factor(arg);
}
};
private final Computable<BigInteger, BigInteger[]> cache
= new Memorizer<BigInteger, BigInteger[]>(c);
public void service(ServletRequest req,
ServletResponse resp) {
try {
BigInteger i = extractFromRequest(req);
encodeIntoResponse(resp, cache.compute(i));
} catch (InterruptedException e) {
encodeError(resp, "factorization interrupted");
}
}
}
SummaryofPartI
We've covered a lot of material so far! The following "concurrency cheat sheet" summarizes the main concepts and rulespresentedinPartI.
x It'sthemutablestate,stupid.[1]
Allconcurrencyissuesboildowntocoordinatingaccesstomutablestate.Thelessmutablestate,theeasieritisto ensurethreadsafety.
x Makefieldsfinalunlesstheyneedtobemutable.
x ImmutableobjectsareautomaticallythreadͲsafe.
Immutableobjectssimplifyconcurrentprogrammingtremendously.Theyaresimplerandsafer,andcanbeshared freelywithoutlockingordefensivecopying.
x Encapsulationmakesitpracticaltomanagethecomplexity.
70 JavaConcurrencyInPractice
You could write a threadͲsafe program with all data stored in global variables, but why would you want to?
Encapsulatingdatawithinobjectsmakesiteasiertopreservetheirinvariants;encapsulatingsynchronizationwithin objectsmakesiteasiertocomplywiththeirsynchronizationpolicy.
x Guardeachmutablevariablewithalock.
x Guardallvariablesinaninvariantwiththesamelock.
x Holdlocksforthedurationofcompoundactions.
x Aprogramthataccessesamutablevariablefrommultiplethreadswithoutsynchronizationisabrokenprogram.
x Don'trelyoncleverreasoningaboutwhyyoudon'tneedtosynchronize.
x IncludethreadsafetyinthedesignprocessorexplicitlydocumentthatyourclassisnotthreadͲsafe.
x Documentyoursynchronizationpolicy.
[1]Duringthe1992U.S.presidentialelection,electoralstrategistJamesCarvillehungasigninBillClinton'scampaignheadquartersreading"The economy,stupid",tokeepthecampaignonmessage.
5BPartII:StructuringConcurrentApplications Ͳ17BChapter5.BuildingBlocks 71
PartII:StructuringConcurrentApplications
Chapter6.TaskExecution
Chapter7.CancellationandShutdown
Chapter8.ApplyingThreadPools
Chapter9.GUIApplications
72 JavaConcurrencyInPractice
Chapter6.TaskExecution
Mostconcurrentapplicationsareorganizedaroundtheexecutionoftasks:abstract,discreteunitsofwork.Dividingthe work of an application into tasks simplifies program organization, facilitates error recovery by providing natural transactionboundaries,andpromotesconcurrencybyprovidinganaturalstructureforparallelizingwork.
6.1.ExecutingTasksinThreads
The first step in organizing a program around task execution is identifying sensible task boundaries. Ideally, tasks are independent activities: work that doesn't depend on the state, result, or side effects of other tasks. Independence facilitatesconcurrency,asindependenttaskscanbeexecutedinparallelifthereareadequateprocessingresources.For greater flexibility in scheduling and load balancing tasks, each task should also represent a small fraction of your application'sprocessingcapacity.
Server applications should exhibit both good throughput and good responsiveness under normal load. Application providers want applications to support as many users as possible, so as to reduce provisioning costs per user; users want to get their response quickly. Further, applications should exhibit graceful degradation as they become overloaded,ratherthansimplyfallingoverunderheavyload.Choosinggoodtaskboundaries,coupledwithasensible taskexecutionpolicy(seeSection6.2.2),canhelpachievethesegoals.
Mostserverapplicationsofferanaturalchoiceoftaskboundary:individualclientrequests.Webservers,mailservers, fileservers,EJBcontainers,anddatabaseserversallacceptrequestsvianetworkconnectionsfromremoteclients.Using individualrequestsastaskboundariesusuallyoffersbothindependenceandappropriatetasksizing.Forexample,the resultofsubmittingamessagetoamailserverisnotaffectedbytheothermessagesbeingprocessedatthesametime, andhandlingasinglemessageusuallyrequiresaverysmallpercentageoftheserver'stotalcapacity.
6.1.1.ExecutingTasksSequentially
Thereareanumberofpossiblepoliciesforschedulingtaskswithinanapplication,someofwhichexploitthepotential forconcurrencybetterthanothers.Thesimplestistoexecutetaskssequentiallyinasinglethread.SingleThreadWebServer in Listing 6.1 processes its tasks Ͳ HTTP requests arriving on port 80 Ͳ sequentially. The details of the request processingaren'timportant;we'reinterestedincharacterizingtheconcurrencyofvariousschedulingpolicies.
Listing6.1.SequentialWebServer.
class SingleThreadWebServer {
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
Socket connection = socket.accept();
handleRequest(connection);
}
}
}
SingleThreadedWebServerissimpleandtheoreticallycorrect,butwouldperformpoorlyinproductionbecauseitcan handle only one request at a time. The main thread alternates between accepting connections and processing the associated request. While the server is handling a request, new connections must wait until it finishes the current request and calls accept again. This might work if request processing were so fast that handleRequest effectively returnedimmediately,butthisdoesn'tdescribeanywebserverintherealworld.
Processing a web request involves a mix of computation and I/O. The server must perform socket I/O to read the request and write the response, which can block due to network congestion or connectivity problems. It may also performfileI/Oormakedatabaserequests,whichcanalsoblock.InasingleͲthreadedserver,blockingnotonlydelays completingthecurrentrequest,butpreventspendingrequestsfrombeingprocessedatall.Ifonerequestblocksforan unusually long time, users might think the server is unavailable because it appears unresponsive. At the same time, resourceutilizationispoor,sincetheCPUsitsidlewhilethesinglethreadwaitsforitsI/Otocomplete.
Inserverapplications,sequentialprocessingrarelyprovideseithergoodthroughputorgoodresponsiveness.Thereare exceptionsͲsuchaswhentasksarefewandlongͲlived,orwhentheserverservesasingleclientthatmakesonlyasingle requestatatimeͲbutmostserverapplicationsdonotworkthisway.[1]
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 73
[1]Insomesituations,sequentialprocessingmayofferasimplicityorsafetyadvantage;mostGUIframeworksprocesstaskssequentiallyusinga singlethread.WereturntothesequentialmodelinChapter9.
6.1.2.ExplicitlyCreatingThreadsforTasks
A more responsive approach is to create a new thread for servicing each request, as shown in ThreadPerTaskWebServerinListing6.2.
Listing6.2.WebServerthatStartsaNewThreadforEachRequest.
class ThreadPerTaskWebServer {
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
final Socket connection = socket.accept();
Runnable task = new Runnable() {
public void run() {
handleRequest(connection);
}
};
new Thread(task).start();
}
}
}
ThreadPerTaskWebServer is similar in structure to the singleͲthreaded version Ͳ the main thread still alternates betweenacceptinganincomingconnectionanddispatchingtherequest.Thedifferenceisthatforeachconnection,the mainloopcreatesanewthreadtoprocesstherequestinsteadofprocessingitwithinthemainthread.Thishasthree mainconsequences:
x Task processing is offloaded from the main thread, enabling the main loop to resume waiting for the next incoming connection more quickly. This enables new connections to be accepted before previous requests complete,improvingresponsiveness.
x Taskscanbeprocessedinparallel,enablingmultiplerequeststobeservicedsimultaneously.Thismayimprove throughputiftherearemultipleprocessors,oriftasksneedtoblockforanyreasonsuchasI/Ocompletion,lock acquisition,orresourceavailability.
x TaskͲhandlingcodemustbethreadͲsafe,becauseitmaybeinvokedconcurrentlyformultipletasks.
Underlighttomoderateload,thethreadͲperͲtaskapproachisanimprovementoversequentialexecution.Aslongas the request arrival rate does not exceed the server's capacity to handle requests, this approach offers better responsivenessandthroughput.
6.1.3.DisadvantagesofUnboundedThreadCreation
For production use, however, the threadͲperͲtask approach has some practical drawbacks, especially when a large numberofthreadsmaybecreated:
Threadlifecycleoverhead.Threadcreationandteardownarenotfree.Theactualoverheadvariesacrossplatforms,but thread creation takes time, introducing latency into request processing, and requires some processing activity by the JVM and OS. If requests are frequent and lightweight, as in most server applications, creating a new thread for each requestcanconsumesignificantcomputingresources.
Resourceconsumption.Activethreadsconsumesystemresources,especiallymemory.Whentherearemorerunnable threads than available processors, threads sit idle. Having many idle threads can tie up a lot of memory, putting pressure on the garbage collector, and having many threads competing for the CPUs can impose other performance costsaswell.IfyouhaveenoughthreadstokeepalltheCPUsbusy,creatingmorethreadswon'thelpandmayeven hurt.
Stability.Thereisalimitonhowmanythreadscanbecreated.Thelimitvariesbyplatformandisaffectedbyfactors includingJVMinvocationparameters,therequestedstacksizeintheThreadconstructor,andlimitsonthreadsplaced bytheunderlyingoperatingsystem.[2]Whenyouhitthislimit,themostlikelyresultisanOutOfMemoryError.tryingto recoverfromsuchanerrorisveryrisky;itisfareasiertostructureyourprogramtoavoidhittingthislimit.
[2]On32Ͳbitmachines,amajorlimitingfactorisaddressspaceforthreadstacks.Eachthreadmaintainstwoexecutionstacks,oneforJavacode andonefornativecode.TypicalJVMdefaultsyieldacombinedstacksizeofaroundhalfamegabyte.(Youcanchangethiswiththe-XssJVMflag orthroughtheThreadconstructor.)IfyoudividetheperͲthreadstacksizeinto232,yougetalimitofafewthousandsortensofthousandsof threads.Otherfactors,suchasOSlimitations,mayimposestricterlimits.
74 JavaConcurrencyInPractice
Uptoacertainpoint,morethreadscanimprovethroughput,butbeyondthatpointcreatingmorethreadsjustslows downyourapplication,andcreatingonethreadtoomanycancauseyourentireapplicationtocrashhorribly.Theway tostayoutofdangeristoplacesomeboundonhowmanythreadsyourapplicationcreates,andtotestyourapplication thoroughlytoensurethat,evenwhenthisboundisreached,itdoesnotrunoutofresources.
The problem with the threadͲperͲtask approach is that nothing places any limit on the number of threads created except the rate at which remote users can throw HTTP requests at it. Like other concurrency hazards, unbounded threadcreationmayappeartoworkjustfineduringprototypinganddevelopment,withproblemssurfacingonlywhen theapplicationisdeployedandunderheavyload.Soamalicioususer,orenoughordinaryusers,canmakeyourweb servercrashifthetrafficloadeverreachesacertainthreshold.Foraserverapplicationthatissupposedtoprovidehigh availabilityandgracefuldegradationunderload,thisisaseriousfailing.
6.2.TheExecutorFramework
Tasksarelogicalunitsofwork,andthreadsareamechanismbywhichtaskscanrunasynchronously.We'veexamined twopoliciesforexecutingtasksusingthreadsͲexecutetaskssequentiallyinasinglethread,andexecuteeachtaskinits ownthread.Bothhaveseriouslimitations:thesequentialapproachsuffersfrompoorresponsivenessandthroughput, andthethreadͲperͲtaskapproachsuffersfrompoorresourcemanagement.
InChapter5,wesawhowtouseboundedqueuestopreventanoverloadedapplicationfromrunningoutofmemory.
Thread pools offer the same benefit for thread management, and java.util.concurrent provides a flexible thread poolimplementationaspartoftheExecutorframework.TheprimaryabstractionfortaskexecutionintheJavaclass librariesisnotThread,butExecutor,showninListing6.3.
Listing6.3. ExecutorInterface.
public interface Executor {
void execute(Runnable command);
}
Executormaybeasimpleinterface,butitformsthebasisforaflexibleandpowerfulframeworkforasynchronoustask execution that supports a wide variety of task execution policies. It provides a standard means of decoupling task submissionfromtaskexecution,describingtaskswithRunnable.TheExecutorimplementationsalsoprovidelifecycle supportandhooksforaddingstatisticsgathering,applicationmanagement,andmonitoring.
ExecutorisbasedontheproducerͲconsumerpattern,whereactivitiesthatsubmittasksaretheproducers(producing unitsofworktobedone)andthethreadsthatexecutetasksaretheconsumers(consumingthoseunitsofwork).Using anExecutorisusuallytheeasiestpathtoimplementingaproducerͲconsumerdesigninyourapplication.
6.2.1.Example:WebServerUsingExecutor
BuildingawebserverwithanExecutoriseasy.TaskExecutionWebServerinListing6.4replacesthehardͲcodedthread creationwithanExecutor.Inthiscase,weuseoneofthestandardExecutorimplementations,afixedͲsizethreadpool with100threads.
In TaskExecutionWebServer, submission of the requestͲhandling task is decoupled from its execution using an Executor, and its behavior can be changed merely by substituting a different Executor implementation. Changing Executor implementations or configuration is far less invasive than changing the way tasks are submitted; Executor configurationisgenerallyaoneͲtimeeventandcaneasilybeexposedfordeploymentͲtimeconfiguration,whereastask submissioncodetendstobestrewnthroughouttheprogramandhardertoexpose.
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 75
Listing6.4.WebServerUsingaThreadPool.
class TaskExecutionWebServer {
private static final int NTHREADS = 100;
private static final Executor exec
= Executors.newFixedThreadPool(NTHREADS);
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(80);
while (true) {
final Socket connection = socket.accept();
Runnable task = new Runnable() {
public void run() {
handleRequest(connection);
}
};
exec.execute(task);
}
}
}
WecaneasilymodifyTaskExecutionWebServertobehavelikeThreadPer-TaskWebServerbysubstitutinganExecutor thatcreatesanewthreadforeachrequest.WritingsuchanExecutoristrivial,asshowninThreadPerTaskExecutorin Listing6.5.
Listing6.5. ExecutorthatStartsaNewThreadforEachTask.
public class ThreadPerTaskExecutor implements Executor {
public void execute(Runnable r) {
new Thread(r).start();
};
}
Similarly, it is also easy to write an Executor that would make TaskExecutionWebServer behave like the singleͲ
threaded version, executing each task synchronously before returning from execute, as shown in WithinThreadExecutorinListing6.6.
6.2.2.ExecutionPolicies
Thevalueofdecouplingsubmissionfromexecutionisthatitletsyoueasilyspecify,andsubsequentlychangewithout greatdifficulty,theexecutionpolicyforagivenclassoftasks.Anexecutionpolicyspecifiesthe"what,where,when,and how"oftaskexecution,including:
Listing6.6. ExecutorthatExecutesTasksSynchronouslyintheCallingThread.
public class WithinThreadExecutor implements Executor {
public void execute(Runnable r) {
r.run();
};
}
x Inwhatthreadwilltasksbeexecuted?
x Inwhatordershouldtasksbeexecuted(FIFO,LIFO,priorityorder)?
x Howmanytasksmayexecuteconcurrently?
x Howmanytasksmaybequeuedpendingexecution?
x Ifataskhastoberejectedbecausethesystemisoverloaded,whichtaskshouldbeselectedasthevictim,and howshouldtheapplicationbenotified?
x Whatactionsshouldbetakenbeforeorafterexecutingatask?
Execution policies are a resource management tool, and the optimal policy depends on the available computing resourcesandyourqualityͲofͲservicerequirements.Bylimitingthenumberofconcurrenttasks,youcanensurethatthe application does not fail due to resource exhaustion or suffer performance problems due to contention for scarce resources.[3] Separating the specification of execution policy from task submission makes it practical to select an executionpolicyatdeploymenttimethatismatchedtotheavailablehardware.
[3] This is analogous to one of the roles of a transaction monitor in an enterprise application: it can throttle the rate at which transactions are allowedtoproceedsoasnottoexhaustoroverstresslimitedresources.
Wheneveryouseecodeoftheform:
new Thread(runnable).start()
76 JavaConcurrencyInPractice
andyouthinkyoumightatsomepointwantamoreflexibleexecutionpolicy,seriouslyconsiderreplacingitwiththeuse ofanExecutor.
6.2.3.ThreadPools
Athreadpool,asitsnamesuggests,managesahomogeneouspoolofworkerthreads.Athreadpoolistightlyboundto aworkqueueholdingtaskswaitingtobeexecuted.Workerthreadshaveasimplelife:requestthenexttaskfromthe workqueue,executeit,andgobacktowaitingforanothertask.
Executing tasks in pool threads has a number of advantages over the threadͲperͲtask approach. Reusing an existing threadinsteadofcreatinganewoneamortizesthreadcreationandteardowncostsovermultiplerequests.Asanadded bonus,sincetheworkerthreadoftenalreadyexistsatthetimetherequestarrives,thelatencyassociatedwiththread creationdoesnotdelaytaskexecution,thusimprovingresponsiveness.Byproperlytuningthesizeofthethreadpool, youcanhaveenoughthreadstokeeptheprocessorsbusywhilenothavingsomanythatyourapplicationrunsoutof memoryorthrashesduetocompetitionamongthreadsforresources.
Theclasslibraryprovidesaflexiblethreadpoolimplementationalongwithsomeusefulpredefinedconfigurations.You cancreateathreadpoolbycallingoneofthestaticfactorymethodsinExecutors: newFixedThreadPool. A fixedͲsize thread pool creates threads as tasks are submitted, up to the maximum pool size, and then attempts to keep the pool size constant (adding new threads if a thread dies due to an unexpected Exception).
newCachedThreadPool.Acachedthreadpoolhasmoreflexibilitytoreapidlethreadswhenthecurrentsizeofthepool exceedsthedemandforprocessing,andtoaddnewthreadswhendemandincreases,butplacesnoboundsonthesize ofthepool.
newSingleThreadExecutor.AsingleͲthreadedexecutorcreatesasingleworkerthreadtoprocesstasks,replacingitifit diesunexpectedly.Tasksareguaranteedtobeprocessedsequentiallyaccordingtotheorderimposedbythetaskqueue (FIFO,LIFO,priorityorder).[4]
[4]SingleͲthreadedexecutorsalsoprovidesufficientinternalsynchronizationtoguaranteethatanymemorywritesmadebytasksarevisibleto subsequenttasks;thismeansthatobjectscanbesafelyconfinedtothe"taskthread"eventhoughthatthreadmaybereplacedwithanotherfrom timetotime.
newScheduledThreadPool.AfixedͲsizethreadpoolthatsupportsdelayedandperiodictaskexecution,similartoTimer.
(SeeSection6.2.5.)
The newFixedThreadPool and newCachedThreadPool factories return instances of the generalͲpurpose ThreadPoolExecutor,whichcanalsobeuseddirectlytoconstructmorespecializedexecutors.Wediscussthreadpool configurationoptionsindepthinChapter8.
Thewebserverin TaskExecutionWebServerusesanExecutorwithaboundedpoolofworkerthreads.Submittinga taskwithexecuteaddsthetasktotheworkqueue,andtheworkerthreadsrepeatedlydequeuetasksfromthework queueandexecutethem.
SwitchingfromathreadͲperͲtaskpolicytoapoolͲbasedpolicyhasabigeffectonapplicationstability:thewebserver willnolongerfailunderheavyload.[5]Italsodegradesmoregracefully,sinceitdoesnotcreatethousandsofthreads thatcompeteforlimitedCPUandmemoryresources.AndusinganExecutoropensthedoortoallsortsofadditional opportunities for tuning, management, monitoring, logging, error reporting, and other possibilities that would have beenfarmoredifficulttoaddwithoutataskexecutionframework.
[5]Whiletheservermaynotfailduetothecreationoftoomanythreads,ifthetaskarrivalrateexceedsthetaskservicerateforlongenoughitis stillpossible(justharder)torunoutofmemorybecauseofthegrowingqueueofRunnablesawaitingexecution.Thiscanbeaddressedwithin theExecutorframeworkbyusingaboundedworkqueueͲseeSection8.3.2.
6.2.4.ExecutorLifecycle
We'veseenhowtocreateanExecutorbutnothowtoshutonedown.AnExecutorimplementationislikelytocreate threads for processing tasks. But the JVM can't exit until all the (nonͲdaemon) threads have terminated, so failing to shutdownanExecutorcouldpreventtheJVMfromexiting.
BecauseanExecutorprocessestasksasynchronously,atanygiventimethestateofpreviouslysubmittedtasksisnot immediatelyobvious.Somemayhavecompleted,somemaybecurrentlyrunning,andothersmaybequeuedawaiting execution.Inshuttingdownanapplication,thereisaspectrumfromgracefulshutdown(finishwhatyou'vestartedbut don't accept any new work) to abrupt shutdown (turn off the power to the machine room), and various points in
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 77
between.SinceExecutorsprovideaservicetoapplications,theyshouldbeabletobeshutdownaswell,bothgracefully andabruptly,andfeedbackinformationtotheapplicationaboutthestatusoftasksthatwereaffectedbytheshutdown.
Toaddresstheissueofexecutionservicelifecycle,theExecutorServiceinterfaceextendsExecutor,addinganumber of methods for lifecycle management (as well as some convenience methods for task submission). The lifecycle managementmethodsofExecutorServiceareshowninListing6.7.
Listing6.7.LifecycleMethodsinExecutorService.
public interface ExecutorService extends Executor {
void shutdown();
List<Runnable> shutdownNow();
boolean isShutdown();
boolean isTerminated();
boolean awaitTermination(long timeout, TimeUnit unit)
throws InterruptedException;
// ... additional convenience methods for task submission
}
The lifecycle implied by ExecutorService has three states Ͳ running, shutting down, and terminated.
ExecutorServices are initially created in the running state. The shutdown method initiates a graceful shutdown: no new tasks are accepted but previously submitted tasks are allowed to complete Ͳ including those that have not yet begunexecution.TheshutdownNowmethodinitiatesanabruptshutdown:itattemptstocanceloutstandingtasksand doesnotstartanytasksthatarequeuedbutnotbegun.
TaskssubmittedtoanExecutorServiceafterithasbeenshutdownarehandledbytherejectedexecutionhandler(see Section 8.3.3), which might silently discard the task or might cause execute to throw the unchecked RejectedExecutionException. Once all tasks have completed, the ExecutorService transitions to the terminated state.YoucanwaitforanExecutorServicetoreachtheterminatedstatewithawaitTermination,orpollforwhether ithasyetterminatedwithisTerminated.ItiscommontofollowshutdownimmediatelybyawaitTermination,creating the effect of synchronously shutting down the ExecutorService.(Executor shutdown and task cancellation are coveredinmoredetailinChapter7.)
LifecycleWebServer in Listing 6.8 extends our web server with lifecycle support. It can be shut down in two ways: programmaticallybycallingstop,andthroughaclientrequestbysendingthewebserveraspeciallyformattedHTTP
request.
Listing6.8.WebServerwithShutdownSupport.
class LifecycleWebServer {
private final ExecutorService exec = ...;
public void start() throws IOException {
ServerSocket socket = new ServerSocket(80);
while (!exec.isShutdown()) {
try {
final Socket conn = socket.accept();
exec.execute(new Runnable() {
public void run() { handleRequest(conn); }
});
} catch (RejectedExecutionException e) {
if (!exec.isShutdown())
log("task submission rejected", e);
}
}
}
public void stop() { exec.shutdown(); }
void handleRequest(Socket connection) {
Request req = readRequest(connection);
if (isShutdownRequest(req))
stop();
else
dispatchRequest(req);
}
}
6.2.5.DelayedandPeriodicTasks
TheTimerfacilitymanagestheexecutionofdeferred("runthistaskin100ms")andperiodic("runthistaskevery10
ms") tasks. However, Timer has some drawbacks, and ScheduledThreadPoolExecutor should be thought of as its replacement.[6] You can construct a ScheduledThreadPoolExecutor through its constructor or through the newScheduledThreadPoolfactory.
78 JavaConcurrencyInPractice
[6] Timer does have support for scheduling based on absolute, not relative time, so that tasks can be sensitive to changes in the system clock; ScheduledThreadPoolExecutorsupportsonlyrelativetime.
ATimercreatesonlyasinglethreadforexecutingtimertasks.Ifatimertasktakestoolongtorun,thetimingaccuracy ofotherTimerTaskscansuffer.IfarecurringTimerTaskisscheduledtorunevery10msandanotherTimer-Tasktakes 40mstorun,therecurringtaskeither(dependingonwhetheritwasscheduledatfixedrateorfixeddelay)getscalled fourtimesinrapidsuccessionafterthelongͲrunningtaskcompletes,or"misses"fourinvocationscompletely.Scheduled threadpoolsaddressthislimitationbylettingyouprovidemultiplethreadsforexecutingdeferredandperiodictasks.
AnotherproblemwithTimeristhatitbehavespoorlyifaTimerTaskthrowsanuncheckedexception.TheTimerthread doesn'tcatchtheexception,soanuncheckedexceptionthrownfromaTimerTaskterminatesthetimerthread.Timer alsodoesn'tresurrectthethreadinthissituation;instead,iterroneouslyassumestheentireTimerwascancelled.Inthis case,TimerTasksthatarealreadyscheduledbutnotyetexecutedareneverrun,andnewtaskscannotbescheduled.
(Thisproblem,called"threadleakage"isdescribedinSection7.3,alongwithtechniquesforavoidingit.) OutOfTimeinListing6.9illustrateshowaTimercanbecomeconfusedinthismannerand,asconfusionlovescompany, howtheTimersharesitsconfusionwiththenexthaplesscallerthattriestosubmitaTimerTask.Youmightexpectthe program to run for six seconds and exit, but what actually happens is that it terminates after one second with an IllegalStateException whose message text is "Timer already cancelled". ScheduledThreadPoolExecutor deals properlywithillͲbehavedtasks;thereislittlereasontouseTimerinJava5.0orlater.
If you need to build your own scheduling service, you may still be able to take advantage of the library by using a DelayQueue,
a
BlockingQueue
implementation
that
provides
the
scheduling
functionality
of
ScheduledThreadPoolExecutor. A DelayQueue manages a collection of Delayed objects. A Delayed has a delay time associated with it: DelayQueue lets you take an element only if its delay has expired. Objects are returned from a DelayQueueorderedbythetimeassociatedwiththeirdelay.
6.3.FindingExploitableParallelism
TheExecutorframeworkmakesiteasytospecifyanexecutionpolicy,butinordertouseanExecutor,youhavetobe abletodescribeyourtaskasaRunnable.Inmostserverapplications,thereisanobvioustaskboundary:asingleclient request.Butsometimesgoodtaskboundariesarenotquitesoobvious,asinmanydesktopapplications.Theremayalso be exploitable parallelism within a single client request in server applications, as is sometimes the case in database servers.(Forafurtherdiscussionofthecompetingdesignforcesinchoosingtaskboundaries,see[CPJ4.4.1.1].) Listing6.9.ClassIllustratingConfusingTimerBehavior.
public class OutOfTime {
public static void main(String[] args) throws Exception {
Timer timer = new Timer();
timer.schedule(new ThrowTask(), 1);
SECONDS.sleep(1);
timer.schedule(new ThrowTask(), 1);
SECONDS.sleep(5);
}
static class ThrowTask extends TimerTask {
public void run() { throw new RuntimeException(); }
}
}
In this section we develop several versions of a component that admit varying degrees of concurrency. Our sample componentisthepageͲrenderingportionofabrowserapplication,whichtakesapageofHTMLandrendersitintoan image buffer. To keep it simple, we assume that the HTML consists only of marked up text interspersed with image elementswithpreͲspecifieddimensionsandURLs.
6.3.1.Example:SequentialPageRenderer
ThesimplestapproachistoprocesstheHTMLdocumentsequentially.Astextmarkupisencountered,renderitintothe imagebuffer;asimagereferencesareencountered,fetchtheimageoverthenetworkanddrawitintotheimagebuffer aswell.Thisiseasytoimplementandrequirestouchingeachelementoftheinputonlyonce(itdoesn'tevenrequire buffering the document), but is likely to annoy the user, who may have to wait a long time before all the text is rendered.
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 79
Alessannoyingbutstillsequentialapproachinvolvesrenderingthetextelementsfirst,leavingrectangularplaceholders for the images, and after completing the initial pass on the document, going back and downloading the images and drawingthemintotheassociatedplaceholder.ThisapproachisshowninSingleThreadRendererinListing6.10.
DownloadinganimagemostlyinvolveswaitingforI/Otocomplete,andduringthistimetheCPUdoeslittlework.Sothe sequentialapproachmayunderutilizetheCPU,andalsomakestheuserwaitlongerthannecessarytoseethefinished page. We can achieve better utilization and responsiveness by breaking the problem into independent tasks that can executeconcurrently.
Listing6.10.RenderingPageElementsSequentially.
public class SingleThreadRenderer {
void renderPage(CharSequence source) {
renderText(source);
List<ImageData> imageData = new ArrayList<ImageData>(); for (ImageInfo imageInfo : scanForImageInfo(source))
imageData.add(imageInfo.downloadImage());
for (ImageData data : imageData)
renderImage(data);
}
}
6.3.2.ResultǦbearingTasks:CallableandFuture
TheExecutorframeworkusesRunnableasitsbasictaskrepresentation.Runnableisafairlylimitingabstraction;run cannot return a value or throw checked exceptions, although it can have side effects such as writing to a log file or placingaresultinashareddatastructure.
ManytasksareeffectivelydeferredcomputationsͲexecutingadatabasequery,fetchingaresourceoverthenetwork, orcomputingacomplicatedfunction.Forthesetypesoftasks,Callableisabetterabstraction:itexpectsthatthemain entry point, call, will return a value and anticipates that it might throw an exception.[7] Executors includes several utilitymethodsforwrappingothertypesoftasks,includingRunnableandjava.security.PrivilegedAction,witha Callable.
[7]ToexpressanonͲvalueͲreturningtaskwithCallable,useCallable<Void>.
RunnableandCallabledescribeabstractcomputationaltasks.Tasksareusuallyfinite:theyhaveaclearstartingpoint and they eventually terminate. The lifecycle of a task executed by an Executor has four phases: created, submitted, started, and completed. Since tasks can take a long time to run, we also want to be able to cancel a task. In the Executorframework,tasksthathavebeensubmittedbutnotyetstartedcanalwaysbecancelled,andtasksthathave startedcansometimesbecancellediftheyareresponsivetointerruption.Cancellingataskthathasalreadycompleted hasnoeffect.(CancellationiscoveredingreaterdetailinChapter7.)
Future represents the lifecycle of a task and provides methods to test whether the task has completed or been cancelled, retrieve its result, and cancel the task. Callable and Future are shown in Listing 6.11. Implicit in the specification of Future is that task lifecycle can only move forwards, not backwards Ͳ just like the ExecutorService lifecycle.Onceataskiscompleted,itstaysinthatstateforever.
Thebehaviorofgetvariesdependingonthetaskstate(notyetstarted,running,completed).Itreturnsimmediatelyor throws an Exception if the task has already completed, but if not it blocks until the task completes. If the task completes by throwing an exception, get rethrows it wrapped in an ExecutionException; if it was cancelled, get throwsCancellationException.IfgetthrowsExecutionException,theunderlyingexceptioncanberetrievedwith getCause.
80 JavaConcurrencyInPractice
Listing6.11. CallableandFutureInterfaces.
public interface Callable<V> {
V call() throws Exception;
}
public interface Future<V> {
boolean cancel(boolean mayInterruptIfRunning);
boolean isCancelled();
boolean isDone();
V get() throws InterruptedException, ExecutionException,
CancellationException;
V get(long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException,
CancellationException, TimeoutException;
}
There are several ways to create a Future to describe a task. The submit methods in ExecutorService all return a Future,sothatyoucan submita RunnableoraCallabletoanexecutorandgetback aFuture thatcan be usedto retrievetheresultorcancelthetask.YoucanalsoexplicitlyinstantiateaFutureTaskforagivenRunnableorCallable.
(BecauseFutureTaskimplementsRunnable,itcanbesubmittedtoanExecutorforexecutionorexecuteddirectlyby callingitsrunmethod.)
As of Java 6, ExecutorService implementations can override newTaskFor in AbstractExecutorService to control instantiation of the Future corresponding to a submitted Callable or Runnable. The default implementation just createsanewFutureTask,asshowninListing6.12.
Listing6.12.DefaultImplementationofnewTaskForinThreadPoolExecutor.
protected <T> RunnableFuture<T> newTaskFor(Callable<T> task) {
return new FutureTask<T>(task);
}
SubmittingaRunnableorCallabletoanExecutorconstitutesasafepublication(seeSection3.5)oftheRunnableor Callable from the submitting thread to the thread that will eventually execute the task. Similarly, setting the result valueforaFutureconstitutesasafepublicationoftheresultfromthethreadinwhichitwascomputedtoanythread thatretrievesitviaget.
6.3.3.Example:PageRendererwithFuture
Asafirststeptowardsmakingthepagerenderermoreconcurrent,let'sdivideitintotwotasks,onethatrendersthe textandonethatdownloadsalltheimages.(BecauseonetaskislargelyCPUͲboundandtheotherislargelyI/OͲbound, thisapproachmayyieldimprovementsevenonsingleͲCPUsystems.)
Callable and Future can help us express the interaction between these cooperating tasks. In FutureRenderer in Listing 6.13, we create a Callable to download all the images, and submit it to an ExecutorService. This returns a Futuredescribingthetask'sexecution;whenthemaintaskgetstothepointwhereitneedstheimages,itwaitsforthe resultbycallingFuture.get.Ifwe'relucky,theresultswillalreadybereadybythetimeweask;otherwise,atleastwe gotaheadstartondownloadingtheimages.
The stateͲdependent nature of get means that the caller need not be aware of the state of the task, and the safe publicationpropertiesoftasksubmissionandresultretrievalmakethisapproachthreadͲsafe.Theexceptionhandling codesurroundingFuture.getdealswithtwopossibleproblems:thatthetaskencounteredanException,orthethread callinggetwasinterruptedbeforetheresultswereavailable.(SeeSections5.5.2and5.4.) FutureRenderer allows the text to be rendered concurrently with downloading the image data. When all the images aredownloaded,theyarerenderedontothepage.Thisisanimprovementinthattheuserseesaresultquicklyandit exploitssomeparallelism,butwecandoconsiderablybetter.Thereisnoneedforuserstowaitforalltheimagestobe downloaded;theywouldprobablyprefertoseeindividualimagesdrawnastheybecomeavailable.
6.3.4.LimitationsofParallelizingHeterogeneousTasks
Inthelastexample,wetriedtoexecutetwodifferenttypesoftasksinparallelͲdownloadingtheimagesandrendering thepage.Butobtainingsignificantperformanceimprovementsbytryingtoparallelizesequentialheterogeneoustasks canbetricky.
Twopeoplecandividetheworkofcleaningthedinnerdishesfairlyeffectively:onepersonwasheswhiletheotherdries.
However,assigningadifferenttypeoftasktoeachworkerdoesnotscalewell;ifseveralmorepeopleshowup,itisnot obvioushowtheycanhelpwithoutgettinginthewayorsignificantlyrestructuringthedivisionoflabor.Withoutfinding finerͲgrainedparallelismamongsimilartasks,thisapproachwillyielddiminishingreturns.
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 81
Afurtherproblemwithdividingheterogeneoustasksamongmultipleworkersisthatthetasksmayhavedisparatesizes.
IfyoudividetasksAandBbetweentwoworkersbutAtakestentimesaslongasB,you'veonlyspeededupthetotal processby9%.Finally,dividingataskamongmultipleworkersalwaysinvolvessomeamountofcoordinationoverhead; forthedivisiontobeworthwhile,thisoverheadmustbemorethancompensatedbyproductivityimprovementsdueto parallelism.
FutureRenderer uses two tasks: one for rendering text and one for downloading the images. If rendering the text is muchfasterthandownloadingtheimages,asisentirelypossible,theresultingperformanceisnotmuchdifferentfrom thesequentialversion,butthecodeisalotmorecomplicated.Andthebestwecandowithtwothreadsisspeedthings upbyafactoroftwo.Thus,tryingtoincreaseconcurrencybyparallelizingheterogeneousactivitiescanbealotofwork, and there is a limit to how much additional concurrency you can get out of it. (See Sections 11.4.2 and 11.4.3 for anotherexampleofthesamephenomenon.)
Listing6.13.WaitingforImageDownloadwithFuture.
public class FutureRenderer {
private final ExecutorService executor = ...;
void renderPage(CharSequence source) {
final List<ImageInfo> imageInfos = scanForImageInfo(source); Callable<List<ImageData>> task =
new Callable<List<ImageData>>() {
public List<ImageData> call() {
List<ImageData> result
= new ArrayList<ImageData>();
for (ImageInfo imageInfo : imageInfos)
result.add(imageInfo.downloadImage());
return result;
}
};
Future<List<ImageData>> future = executor.submit(task); renderText(source);
try {
List<ImageData> imageData = future.get();
for (ImageData data : imageData)
renderImage(data);
} catch (InterruptedException e) {
// Re-assert the thread's interrupted status
Thread.currentThread().interrupt();
// We don't need the result, so cancel the task too
future.cancel(true);
} catch (ExecutionException e) {
throw launderThrowable(e.getCause());
}
}
}
The real performance payoff of dividing a program's workload into tasks comes when there are a large number of independent,homogeneoustasksthatcanbeprocessedconcurrently.
6.3.5. CompletionService:ExecutorMeetsBlockingQueue
IfyouhaveabatchofcomputationstosubmittoanExecutorandyouwanttoretrievetheirresultsastheybecome available,youcouldretaintheFutureassociatedwitheachtaskandrepeatedlypollforcompletionbycallinggetwitha timeoutofzero.Thisispossible,buttedious.Fortunatelythereisabetterway:acompletionservice.
CompletionServicecombinesthefunctionalityofanExecutorandaBlockingQueue.YoucansubmitCallabletasks toitforexecutionandusethequeueͲlikemethodstakeandpolltoretrievecompletedresults,packagedasFutures, astheybecomeavailable.ExecutorCompletionServiceimplementsCompletionService,delegatingthecomputation toanExecutor.
The implementation of ExecutorCompletionService is quite straightforward. The constructor creates a BlockingQueue to hold the completed results. Future-Task has a done method that is called when the computation completes. When a task is submitted, it is wrapped with a QueueingFuture, a subclassof FutureTask that overrides donetoplacetheresultontheBlockingQueue,asshowninListing6.14.Thetakeandpollmethodsdelegatetothe BlockingQueue,blockingifresultsarenotyetavailable.
82 JavaConcurrencyInPractice
Listing6.14. QueueingFutureClassUsedByExecutorCompletionService.
private class QueueingFuture<V> extends FutureTask<V> {
QueueingFuture(Callable<V> c) { super(c); }
QueueingFuture(Runnable t, V r) { super(t, r); }
protected void done() {
completionQueue.add(this);
}
}
6.3.6.Example:PageRendererwithCompletionService
WecanuseaCompletionServicetoimprovetheperformanceofthepagerendererintwoways:shortertotalruntime andimprovedresponsiveness.Wecancreateaseparatetaskfordownloadingeachimageandexecutetheminathread pool,turningthesequentialdownloadintoaparallelone:thisreducestheamountoftimetodownloadalltheimages.
AndbyfetchingresultsfromtheCompletionServiceandrenderingeachimageassoonasitisavailable,wecangive theuseramoredynamicandresponsiveuserinterface.ThisimplementationisshowninRendererinListing6.15.
Listing6.15.UsingCompletionServicetoRenderPageElementsastheyBecomeAvailable.
public class Renderer {
private final ExecutorService executor;
Renderer(ExecutorService executor) { this.executor = executor; }
void renderPage(CharSequence source) {
final List<ImageInfo> info = scanForImageInfo(source);
CompletionService<ImageData> completionService =
new ExecutorCompletionService<ImageData>(executor);
for (final ImageInfo imageInfo : info)
completionService.submit(new Callable<ImageData>() {
public ImageData call() {
return imageInfo.downloadImage();
}
});
renderText(source);
try {
for (int t = 0, n = info.size(); t < n; t++) {
Future<ImageData> f = completionService.take();
ImageData imageData = f.get();
renderImage(imageData);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
throw launderThrowable(e.getCause());
}
}
}
Multiple ExecutorCompletionServices can share a single Executor, so it is perfectly sensible to create an ExecutorCompletionServicethatisprivatetoaparticularcomputationwhilesharingacommonExecutor.Whenused inthisway,aCompletionServiceactsasahandleforabatchofcomputationsinmuchthesamewaythataFuture actsasahandleforasinglecomputation.ByrememberinghowmanytasksweresubmittedtotheCompletionService andcountinghowmanycompletedresultsareretrieved,youcanknowwhenalltheresultsforagivenbatchhavebeen retrieved,evenifyouuseasharedExecutor.
6.3.7.PlacingTimeLimitsonTasks
Sometimes, if an activity does not complete within a certain amount of time, the result is no longer needed and the activitycanbeabandoned.Forexample,awebapplicationmayfetchitsadvertisementsfromanexternaladserver,but iftheadisnotavailablewithintwoseconds,itinsteaddisplaysadefaultadvertisementsothatadunavailabilitydoes notunderminethesite'sresponsivenessrequirements.Similarly,aportalsitemayfetchdatainparallelfrommultiple datasources,butmaybewillingtowaitonlyacertainamountoftimefordatatobeavailablebeforerenderingthepage withoutit.
Theprimarychallengeinexecutingtaskswithinatimebudgetismakingsurethatyoudon'twaitlongerthanthetime budget to get an answer or find out that one is not forthcoming. The timed version of Future.get supports this requirement:itreturnsassoonastheresultisready,butthrowsTimeoutExceptioniftheresultisnotreadywithinthe timeoutperiod.
5BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution 83
A secondary problem when using timed tasks is to stop them when they run out of time, so they do not waste computingresourcesbycontinuingtocomputearesultthatwillnotbeused.Thiscanbeaccomplishedbyhavingthe taskstrictlymanageitsowntimebudgetandabortifitrunsoutoftime,orbycancellingthetaskifthetimeoutexpires.
Again, Future can help; if a timed get completes with a TimeoutException, you can cancel the task through the Future.Ifthetaskiswrittentobecancellable(seeChapter7),itcanbeterminatedearlysoasnottoconsumeexcessive resources.ThistechniqueisusedinListings6.13and6.16.
Listing 6.16 shows a typical application of a timed Future.get. It generates a composite web page that contains the requested content plus an advertisement fetched from an ad server. It submits the adͲfetching task to an executor, computestherestofthepagecontent,andthenwaitsfortheaduntilitstimebudgetrunsout.[8]Ifthegettimesout,it cancels[9]theadͲfetchingtaskandusesadefaultadvertisementinstead.
[8]Thetimeoutpassedtogetiscomputedbysubtractingthecurrenttimefromthedeadline;thismayinfactyieldanegativenumber,butallthe timedmethodsinjava.util.concurrenttreatnegativetimeoutsaszero,sonoextracodeisneededtodealwiththiscase.
[9]ThetrueparametertoFuture.cancelmeansthatthetaskthreadcanbeinterruptedifthetaskiscurrentlyrunning;seeChapter7.
6.3.8.Example:ATravelReservationsPortal
ThetimeͲbudgetingapproachintheprevioussectioncanbeeasilygeneralizedtoanarbitrarynumberoftasks.Consider atravelreservationportal:theuserenterstraveldatesandrequirementsandtheportalfetchesanddisplaysbidsfrom anumberofairlines,hotelsorcarrentalcompanies.Dependingonthecompany,fetchingabidmightinvolveinvokinga web service, consulting a database, performing an EDI transaction, or some other mechanism. Rather than have the response time for the page be driven by the slowest response, it may be preferable to present only the information available within a given time budget. For providers that do not respond in time, the page could either omit them completelyordisplayaplaceholdersuchas"DidnothearfromAirJavaintime."
Listing6.16.FetchinganAdvertisementwithaTimeBudget.
Page renderPageWithAd() throws InterruptedException {
long endNanos = System.nanoTime() + TIME_BUDGET;
Future<Ad> f = exec.submit(new FetchAdTask());
// Render the page while waiting for the ad
Page page = renderPageBody();
Ad ad;
try {
// Only wait for the remaining time budget
long timeLeft = endNanos - System.nanoTime();
ad = f.get(timeLeft, NANOSECONDS);
} catch (ExecutionException e) {
ad = DEFAULT_AD;
} catch (TimeoutException e) {
ad = DEFAULT_AD;
f.cancel(true);
}
page.setAd(ad);
return page;
}
Fetchingabidfromonecompanyisindependentoffetchingbidsfromanother,sofetchingasinglebidisasensibletask boundarythatallowsbidretrievaltoproceedconcurrently.Itwouldbeeasyenoughtocreatentasks,submitthemtoa thread pool, retain the Futures, and use a timed get to fetch each result sequentially via its Future, but there is an eveneasierwayͲinvokeAll.
Listing 6.17 uses the timed version of invokeAll to submit multiple tasks to an ExecutorService and retrieve the results.TheinvokeAllmethodtakesacollectionoftasksandreturnsacollectionofFutures.Thetwocollectionshave identicalstructures;invokeAlladdstheFuturestothereturnedcollectionintheorderimposedbythetaskcollection's iterator,thusallowingthecallertoassociateaFuturewiththeCallableitrepresents.ThetimedversionofinvokeAll willreturnwhenallthetaskshavecompleted,thecallingthreadisinterrupted,orthetimeoutexpires.Anytasksthat are not complete when the timeout expires are cancelled. On return from invokeAll, each task will have either completednormallyorbeencancelled;theclientcodecancallgetorisCancelledtofindoutwhich.
Summary
Structuring applications around the execution of tasks can simplify development and facilitate concurrency. The Executor framework permits you to decouple task submission from execution policy and supports a rich variety of executionpolicies;wheneveryoufindyourselfcreatingthreadstoperformtasks,considerusinganExecutorinstead.
Tomaximizethebenefitofdecomposinganapplicationintotasks,youmustidentifysensibletaskboundaries.Insome
84 JavaConcurrencyInPractice
applications,theobvioustaskboundariesworkwell,whereasinotherssomeanalysismayberequiredtouncoverfinerͲ
grainedexploitableparallelism.
Listing6.17.RequestingTravelQuotesUnderaTimeBudget.
private class QuoteTask implements Callable<TravelQuote> {
private final TravelCompany company;
private final TravelInfo travelInfo;
...
public TravelQuote call() throws Exception {
return company.solicitQuote(travelInfo);
}
}
public List<TravelQuote> getRankedTravelQuotes(
TravelInfo travelInfo, Set<TravelCompany> companies,
Comparator<TravelQuote> ranking, long time, TimeUnit unit)
throws InterruptedException {
List<QuoteTask> tasks = new ArrayList<QuoteTask>();
for (TravelCompany company : companies)
tasks.add(new QuoteTask(company, travelInfo));
List<Future<TravelQuote>> futures =
exec.invokeAll(tasks, time, unit);
List<TravelQuote> quotes =
new ArrayList<TravelQuote>(tasks.size());
Iterator<QuoteTask> taskIter = tasks.iterator();
for (Future<TravelQuote> f : futures) {
QuoteTask task = taskIter.next();
try {
quotes.add(f.get());
} catch (ExecutionException e) {
quotes.add(task.getFailureQuote(e.getCause()));
} catch (CancellationException e) {
quotes.add(task.getTimeoutQuote(e));
}
}
Collections.sort(quotes, ranking);
return quotes;
}
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 85
Chapter7.CancellationandShutdown
It is easy to start tasks and threads. Most of the time we allow them to decide when to stop by letting them run to completion. Sometimes, however, we want to stop tasks or threads earlier than they would on their own, perhaps becausetheusercancelledanoperationortheapplicationneedstoshutdownquickly.
Gettingtasksandthreadstostopsafely,quickly,andreliablyisnotalwayseasy.Javadoesnotprovideanymechanism forsafelyforcingathreadtostopwhatitisdoing.[1]Instead,itprovidesinterruption,acooperativemechanismthatlets onethreadaskanothertostopwhatitisdoing.
[1]ThedeprecatedThread.stopandsuspendmethodswereanattempttoprovidesuchamechanism,butwerequicklyrealizedtobeseriously flawed
and
should
be
avoided.
See
http://java.sun.com/j2se/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html for an explanation of the problemswiththesemethods.
Thecooperativeapproachisrequiredbecausewerarelywantatask,thread,orservicetostopimmediately,sincethat could leave shared data structures in an inconsistent state. Instead, tasks and services can be coded so that, when requested,theycleanupanyworkcurrentlyinprogressandthenterminate.Thisprovidesgreaterflexibility,sincethe taskcodeitselfisusuallybetterabletoassessthecleanuprequiredthanisthecoderequestingcancellation.
EndͲofͲlifecycle issues can complicate the design and implementation of tasks, services, and applications, and this importantelementofprogramdesignistoooftenignored.Dealingwellwithfailure,shutdown,andcancellationisone of the characteristics that distinguish awellͲbehaved application from one that merely works. This chapter addresses mechanisms for cancellation and interruption, and how to code tasks and services to be responsive to cancellation requests.
7.1.TaskCancellation
Anactivityiscancellableifexternalcodecanmoveittocompletionbeforeitsnormalcompletion.Thereareanumber ofreasonswhyyoumightwanttocancelanactivity:
UserͲrequested cancellation. The user clicked on the "cancel" button in a GUI application, or requested cancellation throughamanagementinterfacesuchasJMX(JavaManagementExtensions).
TimeͲlimited activities. An application searches a problem space for a finite amount of time and chooses the best solutionfoundwithinthattime.Whenthetimerexpires,anytasksstillsearchingarecancelled.
Applicationevents.Anapplicationsearchesaproblemspacebydecomposingitsothatdifferenttaskssearchdifferent regionsoftheproblemspace.Whenonetaskfindsasolution,allothertasksstillsearchingarecancelled.
Errors. A web crawler searches for relevant pages, storing pages or summary data to disk. When a crawler task encountersanerror(forexample,the diskisfull),othercrawlingtasksarecancelled,possiblyrecordingtheircurrent statesothattheycanberestartedlater.
Shutdown. When an application or service is shut down, something must be done about work that is currently being processedorqueuedforprocessing.Inagracefulshutdown,taskscurrentlyinprogressmightbeallowedtocomplete; inamoreimmediateshutdown,currentlyexecutingtasksmightbecancelled.
ThereisnosafewaytopreemptivelystopathreadinJava,andthereforenosafewaytopreemptivelystopatask.There are only cooperative mechanisms, by which the task and the code requesting cancellation follow an agreedͲupon protocol.
Onesuchcooperativemechanismissettinga"cancellationrequested"flagthatthetaskchecksperiodically;ifitfinds the flag set, the task terminates early. PrimeGenerator in Listing 7.1, which enumerates prime numbers until it is cancelled, illustrates this technique. The cancel method sets the cancelled flag, and the main loop polls this flag beforesearchingforthenextprimenumber.(Forthistoworkreliably,cancelledmustbevolatile.) Listing7.2showsasampleuseofthisclassthatletstheprimegeneratorrunforonesecondbeforecancellingit.The generator won't necessarily stop after exactly one second, since there may be some delay between the time that cancellationisrequestedandthetimethattherunloopnextchecksforcancellation.Thecancelmethodiscalledfrom afinallyblocktoensurethattheprimegeneratoriscancelledevenifthecalltosleepisinterrupted.Ifcancelwere notcalled,theprimeͲseekingthreadwouldrunforever,consumingCPUcyclesandpreventingtheJVMfromexiting.
86 JavaConcurrencyInPractice
A task that wants to be cancellable must have a cancellation policy that specifies the "how", "when", and "what" of cancellationͲhowothercodecanrequestcancellation,whenthetaskcheckswhethercancellationhasbeenrequested, andwhatactionsthetasktakesinresponsetoacancellationrequest.
Consider the realͲworld example of stopping payment on a check. Banks have rules about how to submit a stopͲ
paymentrequest,whatresponsivenessguaranteesitmakesinprocessingsuchrequests,andwhatproceduresitfollows when payment is actually stopped (such as notifying the other bank involved in the transaction and assessing a fee against the payer�’s account). Taken together, these procedures and guarantees comprise the cancellation policy for checkpayment.
Listing7.1.UsingaVolatileFieldtoHoldCancellationState.
@ThreadSafe
public class PrimeGenerator implements Runnable {
@GuardedBy("this")
private final List<BigInteger> primes
= new ArrayList<BigInteger>();
private volatile boolean cancelled;
public void run() {
BigInteger p = BigInteger.ONE;
while (!cancelled ) {
p = p.nextProbablePrime();
synchronized (this) {
primes.add(p);
}
}
}
public void cancel() { cancelled = true; }
public synchronized List<BigInteger> get() {
return new ArrayList<BigInteger>(primes);
}
}
Listing7.2.GeneratingaSecond'sWorthofPrimeNumbers.
List<BigInteger> aSecondOfPrimes() throws InterruptedException {
PrimeGenerator generator = new PrimeGenerator();
new Thread(generator).start();
try {
SECONDS.sleep(1);
} finally {
generator.cancel();
}
return generator.get();
}
PrimeGeneratorusesasimplecancellationpolicy:clientcoderequestscancellationbycallingcancel,PrimeGenerator checksforcancellationonceperprimefoundandexitswhenitdetectscancellationhasbeenrequested.
7.1.1.Interruption
ThecancellationmechanisminPrimeGeneratorwilleventuallycausetheprimeͲseekingtasktoexit,butitmighttakea while.If,however,ataskthatusesthisapproachcallsablockingmethodsuchasBlockingQueue.put,wecouldhavea moreseriousproblemͲthetaskmightnevercheckthecancellationflagandthereforemightneverterminate.
BrokenPrimeProducerinListing7.3illustratesthisproblem.Theproducerthreadgeneratesprimesandplacesthemon ablockingqueue.Iftheproducergetsaheadoftheconsumer,thequeuewillfillupandputwillblock.Whathappensif theconsumertriestocanceltheproducertaskwhileitisblockedinput?Itcancallcancelwhichwillsetthecancelled flag Ͳ but the producer will never check the flag because it will never emerge from the blocking put (because the consumerhasstoppedretrievingprimesfromthequeue).
AswehintedinChapter5,certainblockinglibrarymethodssupportinterruption.Threadinterruptionisacooperative mechanismforathreadtosignalanotherthreadthatitshould,atitsconvenienceandifitfeelslikeit,stopwhatitis doinganddosomethingelse.
ThereisnothingintheAPIorlanguagespecificationthattiesinterruptiontoanyspecificcancellationsemantics,butin practice,usinginterruptionforanythingbutcancellationisfragileanddifficulttosustaininlargerapplications.
Eachthreadhasabooleaninterruptedstatus;interruptingathreadsetsitsinterruptedstatustotrue.Threadcontains methods for interrupting a thread and querying the interrupted status of a thread, as shown in Listing 7.4. The
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 87
interruptmethodinterruptsthetargetthread,andisInterruptedreturnstheinterruptedstatusofthetargetthread.
The poorly named static interrupted method clears the interrupted status of the current thread and returns its previousvalue;thisistheonlywaytocleartheinterruptedstatus.
Blocking library methods like Thread.sleep and Object.wait try to detect when a thread has been interrupted and return early. They respond to interruption by clearing the interrupted status and throwing InterruptedException, indicating that the blocking operation completed early due to interruption. The JVM makes no guarantees on how quicklyablockingmethodwilldetectinterruption,butinpracticethishappensreasonablyquickly.
Listing7.3.UnreliableCancellationthatcanLeaveProducersStuckinaBlockingOperation.
class BrokenPrimeProducer extends Thread {
private final BlockingQueue<BigInteger> queue;
private volatile boolean cancelled = false;
BrokenPrimeProducer(BlockingQueue<BigInteger> queue) {
this.queue = queue;
}
public void run() {
try {
BigInteger p = BigInteger.ONE;
while (!cancelled)
queue.put(p = p.nextProbablePrime());
} catch (InterruptedException consumed) { }
}
public void cancel() { cancelled = true; }
}
void consumePrimes() throws InterruptedException {
BlockingQueue<BigInteger> primes = ...;
BrokenPrimeProducer producer = new BrokenPrimeProducer(primes);
producer.start();
try {
while (needMorePrimes())
consume(primes.take());
} finally {
producer.cancel();
}
}
Listing7.4.InterruptionMethodsinThread.
public class Thread {
public void interrupt() { ... }
public boolean isInterrupted() { ... }
public static boolean interrupted() { ... }
...
}
Ifathreadisinterruptedwhenitisnotblocked,itsinterruptedstatusisset,anditisuptotheactivitybeingcancelledto poll the interrupted status to detect interruption. In this way interruption is "sticky�” if it doesn't trigger an InterruptedException,evidenceofinterruptionpersistsuntilsomeonedeliberatelyclearstheinterruptedstatus.
Calling interrupt does not necessarily stop the target thread from doing what it is doing; it merely delivers the messagethatinterruptionhasbeenrequested.
Agoodwaytothinkaboutinterruptionisthatitdoesnotactuallyinterruptarunningthread;itjustrequeststhatthe thread interrupt itself at the next convenient opportunity. (These opportunities are called cancellation points.) Some methods, such as wait, sleep, and join, take such requests seriously, throwing an exception when they receive an interrupt request or encounter an already set interrupt status upon entry. Well behaved methods may totally ignore such requests so long as they leave the interruption request in place so that calling code can do something with it.
Poorlybehavedmethodsswallowtheinterruptrequest,thusdenyingcodefurtherupthecallstacktheopportunityto actonit.
Thestaticinterruptedmethodshouldbeusedwithcaution,becauseitclearsthecurrentthread'sinterruptedstatus.If youcallinterruptedanditreturnsTRue,unlessyouareplanningtoswallowtheinterruption,youshoulddosomething
88 JavaConcurrencyInPractice
withitͲeitherthrowInterruptedExceptionorrestoretheinterruptedstatusbycallinginterruptagain,asinListing 5.10onpage94.
BrokenPrimeProducerillustrateshowcustomcancellationmechanismsdonotalwaysinteractwellwithblockinglibrary methods. If you code your tasks to be responsive to interruption, you can use interruption as your cancellation mechanismandtakeadvantageoftheinterruptionsupportprovidedbymanylibraryclasses.
Interruptionisusuallythemostsensiblewaytoimplementcancellation.
BrokenPrimeProducer can be easily fixed (and simplified) by using interruption instead of a boolean flag to request cancellation,asshowninListing7.5.Therearetwopointsineachloopiterationwhereinterruptionmaybedetected:in theblockingputcall,andbyexplicitlypollingtheinterruptedstatusintheloopheader.Theexplicittestisnotstrictly necessaryherebecauseoftheblockingputcall,butitmakesPrimeProducermoreresponsivetointerruptionbecause it checks for interruption before starting the lengthy task of searching for a prime, rather than after. When calls to interruptible blocking methods are not frequent enough to deliver the desired responsiveness, explicitly testing the interruptedstatuscanhelp.
Listing7.5.UsingInterruptionforCancellation.
class PrimeProducer extends Thread {
private final BlockingQueue<BigInteger> queue;
PrimeProducer(BlockingQueue<BigInteger> queue) {
this.queue = queue;
}
public void run() {
try {
BigInteger p = BigInteger.ONE;
while (!Thread.currentThread().isInterrupted())
queue.put(p = p.nextProbablePrime());
} catch (InterruptedException consumed) {
/* Allow thread to exit */
}
}
public void cancel() { interrupt(); }
}
7.1.2.InterruptionPolicies
Just as tasks should have a cancellation policy, threads should have an interruption policy. An interruption policy determineshowathreadinterpretsaninterruptionrequestͲwhatitdoes(ifanything)whenoneisdetected,whatunits ofworkareconsideredatomicwithrespecttointerruption,andhowquicklyitreactstointerruption.
The most sensible interruption policy is some form of threadͲlevel or serviceͲlevel cancellation: exit as quickly as practical,cleaningupifnecessary,andpossiblynotifyingsomeowningentitythatthethreadisexiting.Itispossibleto establish other interruption policies, such as pausing or resuming a service, but threads or thread pools with nonstandardinterruptionpoliciesmayneedtoberestrictedtotasksthathavebeenwrittenwithanawarenessofthe policy.
It is important to distinguish between how tasks and threads should react to interruption. A single interrupt request may have more than one desired recipient interrupting a worker thread in a thread pool can mean both "cancel the currenttask"and"shutdowntheworkerthread".
Tasksdonotexecuteinthreadstheyown;theyborrowthreadsownedbyaservicesuchasathreadpool.Codethat doesn'townthethread(forathreadpool,anycodeoutsideofthethreadpoolimplementation)shouldbecarefulto preservetheinterruptedstatussothattheowningcodecaneventuallyactonit,evenifthe"guest"codeactsonthe interruptionaswell.(IfyouarehouseͲsittingforsomeone,youdon'tthrowoutthemailthatcomeswhilethey'reawayͲ
yousaveitandletthemdealwithitwhentheygetback,evenifyoudoreadtheirmagazines.) ThisiswhymostblockinglibrarymethodssimplythrowInterruptedExceptioninresponsetoaninterrupt.Theywill neverexecuteinathreadtheyown,sotheyimplementthemostreasonablecancellationpolicyfortaskorlibrarycode: getoutofthewayasquicklyaspossibleandcommunicatetheinterruptionbacktothecallersothatcodehigherupon thecallstackcantakefurtheraction.
Ataskneedn'tnecessarilydropeverythingwhenitdetectsaninterruptionrequestͲitcanchoosetopostponeituntila moreopportunetimebyrememberingthatitwasinterrupted,finishingthetaskitwasperforming,andthenthrowing
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 89
InterruptedException or otherwise indicating interruption. This technique can protect data structures from corruptionwhenanactivityisinterruptedinthemiddleofanupdate.
Ataskshouldnotassumeanythingabouttheinterruptionpolicyofitsexecutingthreadunlessitisexplicitlydesignedto run within a service that has a specific interruption policy. Whether a task interprets interruption as cancellation or takessomeotheractiononinterruption,itshouldtakecaretopreservetheexecutingthread'sinterruptionstatus.Ifitis not simply going to propagate InterruptedException to its caller, it should restore the interruption status after catchingInterruptedException:
Thread.currentThread().interrupt();
Just as task code should not make assumptions about what interruption means to its executing thread, cancellation codeshouldnotmakeassumptionsabouttheinterruptionpolicyofarbitrarythreads.Athreadshouldbeinterrupted only by its owner; the owner can encapsulate knowledge of the thread's interruption policy in an appropriate cancellationmechanismsuchasashutdownmethod.
Because each thread has its own interruption policy, you should not interrupt a thread unless you know what interruptionmeanstothatthread.
CriticshavederidedtheJavainterruptionfacilitybecauseitdoesnotprovideapreemptiveinterruptioncapabilityand yet forces developers to handle InterruptedException. However, the ability to postpone an interruption request enablesdeveloperstocraftflexibleinterruptionpoliciesthatbalanceresponsivenessandrobustnessasappropriatefor theapplication.
7.1.3.RespondingtoInterruption
As mentioned in Section 5.4, when you call an interruptible blocking method such as Thread.sleep or BlockingQueue.put,therearetwopracticalstrategiesforhandlingInterruptedException: x Propagate the exception (possibly after some taskͲspecific cleanup), making your method an interruptible blockingmethod,too;or
x Restoretheinterruptionstatussothatcodehigheruponthecallstackcandealwithit.
PropagatingInterruptedExceptioncanbeaseasyasaddingInterruptedExceptiontothethrowsclause,asshown bygetNextTaskinListing7.6.
Listing7.6.PropagatingInterruptedExceptiontoCallers.
BlockingQueue<Task> queue;
...
public Task getNextTask() throws InterruptedException {
return queue.take();
}
Ifyoudon'twanttoorcannotpropagateInterruptedException(perhapsbecauseyourtaskisdefinedbyaRunnable), you need to find another way to preserve the interruption request. The standard way to do this is to restore the interrupted status by calling interrupt again. What you should not do is swallow the InterruptedException by catchingitanddoingnothinginthecatchblock,unlessyourcodeisactuallyimplementingtheinterruptionpolicyfora thread.PrimeProducerswallowstheinterrupt,butdoessowiththeknowledgethatthethreadisabouttoterminate andthatthereforethereisnocodehigheruponthecallstackthatneedstoknowabouttheinterruption.Mostcode doesnotknowwhatthreaditwillruninandsoshouldpreservetheinterruptedstatus.
Only code that implements a thread's interruption policy may swallow an interruption request. GeneralͲpurpose task andlibrarycodeshouldneverswallowinterruptionrequests.
Activitiesthatdonotsupportcancellationbutstillcallinterruptibleblockingmethodswillhavetocalltheminaloop, retryingwheninterruptionisdetected.Inthiscase,theyshouldsavetheinterruptionstatuslocallyandrestoreitjust beforereturning,asshowninListing 7.7,ratherthanimmediatelyuponcatching InterruptedException.Settingthe interrupted status too early could result in an infinite loop, because most interruptible blocking methods check the interrupted status on entry and throw InterruptedException immediately if it is set. (Interruptible methods usually pollforinterruptionbeforeblockingordoinganysignificantwork,soastobeasresponsivetointerruptionaspossible.) Ifyourcodedoesnotcallinterruptibleblockingmethods,itcanstillbemaderesponsivetointerruptionbypollingthe current thread's interrupted status throughout the task code. Choosing a polling frequency is a tradeoff between
90 JavaConcurrencyInPractice
efficiencyandresponsiveness.Ifyouhavehighresponsivenessrequirements,youcannotcallpotentiallylongͲrunning methodsthatarenotthemselvesresponsivetointerruption,potentiallyrestrictingyouroptionsforcallinglibrarycode.
Cancellationcaninvolvestateotherthantheinterruptionstatus;interruptioncanbeusedtogetthethread'sattention, and information stored elsewhere by the interrupting thread can be used to provide further instructions for the interruptedthread.(Besuretousesynchronizationwhenaccessingthatinformation.) Listing7.7.NonǦcancelableTaskthatRestoresInterruptionBeforeExit.
public Task getNextTask(BlockingQueue<Taskgt; queue) {
boolean interrupted = false;
try {
while (true) {
try {
return queue.take();
} catch (InterruptedException e) {
interrupted = true;
// fall through and retry
}
}
} finally {
if (interrupted)
Thread.currentThread().interrupt();
}
}
Forexample,whenaworkerthreadownedbyaThreadPoolExecutordetectsinterruption,itcheckswhetherthepool isbeingshutdown.Ifso,itperformssomepoolcleanupbeforeterminating;otherwiseitmaycreateanewthreadto restorethethreadpooltothedesiredsize.
7.1.4.Example:TimedRun
Many problems can take forever to solve (e.g., enumerate all the prime numbers); for others, the answer might be found reasonably quickly but also might take forever. Being able to say "spend up to ten minutes looking for the answer"or"enumeratealltheanswersyoucanintenminutes"canbeusefulinthesesituations.
The aSecondOfPrimes method in Listing 7.2 starts a PrimeGenerator and interrupts it after a second. While the PrimeGenerator might take somewhat longer than a second to stop, it will eventually notice the interrupt and stop, allowingthethreadtoterminate.Butanotheraspectofexecutingataskisthatyouwanttofindoutifthetaskthrows an exception. If PrimeGenerator throws an unchecked exception before the timeout expires, it will probably go unnoticed,sincetheprimegeneratorrunsinaseparatethreadthatdoesnotexplicitlyhandleexceptions.
Listing7.8showsanattemptatrunninganarbitraryRunnableforagivenamountoftime.Itrunsthetaskinthecalling thread and schedules a cancellation task to interrupt it after a given time interval. This addresses the problem of uncheckedexceptionsthrownfromthetask,sincetheycanthenbecaughtbythecalleroftimedRun.
Thisisanappealinglysimpleapproach,butitviolatestherules:youshouldknowathread'sinterruptionpolicybefore interruptingit.SincetimedRuncanbecalledfromanarbitrarythread,itcannotknowthecallingthread'sinterruption policy.Ifthetaskcompletesbeforethetimeout,thecancellationtaskthatinterruptsthethreadinwhichtimedRunwas called could go off after timedRun has returned to its caller. We don't know what code will be running when that happens, but the result won't be good. (It is possible but surprisingly tricky to eliminate this risk by using the ScheduledFuturereturnedbyscheduletocancelthecancellationtask.)
Listing7.8.SchedulinganInterruptonaBorrowedThread.
private static final ScheduledExecutorService cancelExec = ...;
public static void timedRun(Runnable r,
long timeout, TimeUnit unit) {
final Thread taskThread = Thread.currentThread();
cancelExec.schedule(new Runnable() {
public void run() { taskThread.interrupt(); }
}, timeout, unit);
r.run();
}
Further,ifthetaskisnotresponsivetointerruption,timedRunwillnotreturnuntilthetaskfinishes,whichmaybelong afterthedesiredtimeout(orevennotatall).Atimedrunservicethatdoesn'treturnafterthespecifiedtimeislikelyto beirritatingtoitscallers.
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 91
Listing7.9addressestheexceptionͲhandlingproblemofaSecondOfPrimesandtheproblemswiththepreviousattempt.
The thread created to run the task can have its own execution policy, and even if the task doesn't respond to the interrupt,thetimedrunmethodcanstillreturntoitscaller.Afterstartingthetaskthread,timedRunexecutesatimed joinwiththenewlycreatedthread.Afterjoinreturns,itchecksifanexceptionwasthrownfromthetaskandifso, rethrowsitinthethreadcallingtimedRun.ThesavedThrowableissharedbetweenthetwothreads,andsoisdeclared volatiletosafelypublishitfromthetaskthreadtothetimedRunthread.
This version addresses the problems in the previous examples, but because it relies on a timed join, it shares a deficiencywithjoin:wedon'tknowifcontrolwasreturnedbecausethethreadexitednormallyorbecausethejoin timedout.[2]
[2]ThisisaflawintheThreadAPI,becausewhetherornotthejoincompletessuccessfullyhasmemoryvisibilityconsequencesintheJava MemoryModel,butjoindoesnotreturnastatusindicatingwhetheritwassuccessful.
7.1.5.CancellationViaFuture
We've already used an abstraction for managing the lifecycle of a task, dealing with exceptions, and facilitating cancellationFuture.Followingthegeneralprinciplethatitisbettertouseexistinglibraryclassesthantorollyourown, let'sbuildtimedRunusingFutureandthetaskexecutionframework.
Listing7.9.InterruptingaTaskinaDedicatedThread.
public static void timedRun(final Runnable r,
long timeout, TimeUnit unit)
throws InterruptedException {
class RethrowableTask implements Runnable {
private volatile Throwable t;
public void run() {
try { r.run(); }
catch (Throwable t) { this.t = t; }
}
void rethrow() {
if (t != null)
throw launderThrowable(t);
}
}
RethrowableTask task = new RethrowableTask();
final Thread taskThread = new Thread(task);
taskThread.start();
cancelExec.schedule(new Runnable() {
public void run() { taskThread.interrupt(); }
}, timeout, unit);
taskThread.join(unit.toMillis(timeout));
task.rethrow();
}
ExecutorService.submit returns a Future describing the task. Future has a cancel method that takes a boolean argument, mayInterruptIfRunning, and returns a value indicating whether the cancellation attempt was successful.
(This tells you only whether it was able to deliver the interruption, not whether the task detected and acted on it.) When mayInterruptIfRunning is true and the task is currently running in some thread, then that thread is interrupted.Settingthisargumenttofalsemeans"don'trunthistaskifithasn'tstartedyet",andshouldbeusedfor tasksthatarenotdesignedtohandleinterruption.
Since you shouldn't interrupt a thread unless you know its interruption policy, when is it OK to call cancel with an argument of TRue? The task execution threads created by the standard Executor implementations implement an interruption policy that lets tasks be cancelled using interruption, so it is safe to set mayInterruptIfRunning when cancellingtasksthroughtheirFutureswhentheyarerunninginastandardExecutor.Youshouldnotinterruptapool thread directly when attempting to cancel a task, because you won't know what task is running when the interrupt request is delivered Ͳ do this only through the task's Future. This is yet another reason to code tasks to treat interruptionasacancellationrequest:thentheycanbecancelledthroughtheirFutures.
Listing7.10showsaversionoftimedRunthatsubmitsthetasktoanExecutorServiceandretrievestheresultwitha timedFuture.get.IfgetterminateswithaTimeoutException,thetaskiscancelledviaitsFuture.(Tosimplifycoding, this version calls Future.cancel unconditionally in a finally block, taking advantage of the fact that cancelling a completedtaskhasnoeffect.)Iftheunderlyingcomputationthrowsanexceptionpriortocancellation,itisrethrown fromtimedRun,whichisthemostconvenientwayforthecallertodealwiththeexception.Listing7.10alsoillustrates
92 JavaConcurrencyInPractice
anothergoodpractice:cancellingtaskswhoseresultisnolongerneeded.(ThistechniquewasalsousedinListing6.13
onpage128andListing6.16onpage132.)
Listing7.10.CancellingaTaskUsingFuture.
public static void timedRun(Runnable r,
long timeout, TimeUnit unit)
throws InterruptedException {
Future<?> task = taskExec.submit(r);
try {
task.get(timeout, unit);
} catch (TimeoutException e) {
// task will be cancelled below
} catch (ExecutionException e) {
// exception thrown in task; rethrow
throw launderThrowable(e.getCause());
} finally {
// Harmless if task already completed
task.cancel(true); // interrupt if running
}
}
When Future.get throws InterruptedException or TimeoutException and you know that the result is no longer neededbytheprogram,cancelthetaskwithFuture.cancel.
7.1.6.DealingwithNonͲinterruptibleBlocking
ManyblockinglibrarymethodsrespondtointerruptionbyreturningearlyandthrowingInterruptedException,which makes it easier to build tasks that are responsive to cancellation. However, not all blocking methods or blocking mechanisms are responsive to interruption; if a thread is blocked performing synchronous socket I/O or waiting to acquire an intrinsic lock, interruption has no effect other than setting the thread's interrupted status. We can sometimes convince threads blocked in nonͲinterruptible activities to stop by means similar to interruption, but this requiresgreaterawarenessofwhythethreadisblocked.
Synchronous socket I/O in java.io. The common form of blocking I/O in server applications is reading or writing to a socket. Unfortunately, the read and write methods in InputStream and OutputStream are not responsive to interruption,butclosingtheunderlyingsocketmakesanythreadsblockedinreadorwritethrowaSocketException.
Synchronous I/O in java.nio. Interrupting a thread waiting on an InterruptibleChannel causes it to throw ClosedByInterruptException and close the channel (and also causes all other threads blocked on the channel to throw ClosedByInterruptException). Closing an InterruptibleChannel causes threads blocked on channel operationstothrowAsynchronousCloseException.MoststandardChannelsimplementInterruptibleChannel.
AsynchronousI/OwithSelector.IfathreadisblockedinSelector.select(injava.nio.channels),wakeupcausesitto returnprematurelybythrowingaClosedSelectorException.
Lock acquisition. If a thread is blocked waiting for an intrinsic lock, there is nothing you can do to stop it short of ensuringthatiteventuallyacquiresthelockandmakesenoughprogressthatyoucangetitsattentionsomeotherway.
However,theexplicitLockclassesofferthelockInterruptiblymethod,whichallowsyoutowaitforalockandstillbe responsivetointerruptsͲseeChapter13.
ReaderThreadinListing7.11showsatechniqueforencapsulatingnonstandardcancellation.ReaderThreadmanagesa singlesocketconnection,readingsynchronouslyfromthesocketandpassinganydatareceivedtoprocessBuffer.To facilitateterminatingauserconnectionorshuttingdowntheserver,ReaderThreadoverridesinterrupttobothdeliver astandardinterruptandclosetheunderlyingsocket;thusinterruptingaReaderThreadmakesitstopwhatitisdoing whetheritisblockedinreadorinaninterruptibleblockingmethod.
7.1.7.EncapsulatingNonstandardCancellationwithNewtaskfor
The technique used in ReaderThread to encapsulate nonstandard cancellation can be refined using the newTaskFor hookaddedtoThreadPoolExecutorinJava6.WhenaCallableissubmittedtoanExecutorService,submitreturns a Future that can be used to cancel the task. The newTaskFor hook is a factory method that creates the Future representing the task. It returns a RunnableFuture, an interface that extends both Future and Runnable (and is implementedbyFutureTask).
CustomizingthetaskFutureallowsyoutooverrideFuture.cancel.Customcancellationcodecanperformloggingor gather statistics on cancellation, and can also be used to cancel activities that are not responsive to interruption.
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 93
ReaderThread encapsulates cancellation of socketͲusing threads by overriding interrupt; the same can be done for tasksbyoverridingFuture.cancel.
CancellableTask in Listing 7.12 defines a CancellableTask interface that extends Callable and adds a cancel method and a newTask factory method for constructing a RunnableFuture. CancellingExecutor extends ThreadPoolExecutor,andoverridesnewTaskFortoletaCancellableTaskcreateitsownFuture.
Listing7.11.EncapsulatingNonstandardCancellationinaThreadbyOverridingInterrupt.
public class ReaderThread extends Thread {
private final Socket socket;
private final InputStream in;
public ReaderThread(Socket socket) throws IOException {
this.socket = socket;
this.in = socket.getInputStream();
}
public void interrupt() {
try {
socket.close();
}
catch (IOException ignored) { }
finally {
super.interrupt();
}
}
public void run() {
try {
byte[] buf = new byte[BUFSZ];
while (true) {
int count = in.read(buf);
if (count < 0)
break;
else if (count > 0)
processBuffer(buf, count);
}
} catch (IOException e) { /* Allow thread to exit */ }
}
}
SocketUsingTask implements CancellableTask and defines Future.cancel to close the socket as well as call super.cancel.IfaSocketUsingTaskiscancelledthroughitsFuture,thesocketisclosedandtheexecutingthreadis interrupted. This increases the task's responsiveness to cancellation: not only can it safely call interruptible blocking methodswhileremainingresponsivetocancellation,butitcanalsocallblockingsocketI/Omethods.
7.2.StoppingaThreadǦbasedService
Applications commonly create services that own threads, such as thread pools, and the lifetime of these services is usually longer than that of the method that creates them. If the application is to shut down gracefully, the threads ownedbytheseservicesneedtobeterminated.Sincethereisnopreemptivewaytostopathread,theymustinstead bepersuadedtoshutdownontheirown.
SensibleencapsulationpracticesdictatethatyoushouldnotmanipulateathreadͲinterruptit,modifyitspriority,etc.Ͳ
unlessyouownit.ThethreadAPIhasnoformalconceptofthreadownership:athreadisrepresentedwitha Thread objectthatcanbefreelysharedlikeanyotherobject.However,itmakessensetothinkofathreadashavinganowner, andthisisusuallytheclassthatcreatedthethread.Soathreadpoolownsitsworkerthreads,andifthosethreadsneed tobeinterrupted,thethreadpoolshouldtakecareofit.
Aswithanyotherencapsulatedobject,threadownershipisnottransitive:theapplicationmayowntheserviceandthe service may own the worker threads, but the application doesn't own the worker threads and therefore should not attempttostopthemdirectly.Instead,theserviceshouldprovidelifecyclemethodsforshuttingitselfdownthatalso shut down the owned threads; then the application can shut down the service, and the service can shut down the threads. ExecutorService provides the shutdown and shutdownNow methods; other threadͲowning services should provideasimilarshutdownmechanism.
ProvidelifecyclemethodswheneverathreadͲowningservicehasalifetimelongerthanthatofthemethodthatcreated it.
94 JavaConcurrencyInPractice
7.2.1.Example:ALoggingService
Most server applications use logging, which can be as simple as inserting println statements into the code. Stream classeslikePrintWriterarethreadͲsafe,sothissimpleapproachwouldrequirenoexplicitsynchronization.[3]However, as we'll see in Section 11.6, inline logging can have some performance costs in highvolume applications. Another alternativeishavethelogcallqueuethelogmessageforprocessingbyanotherthread.
[3] If you are logging multiple lines as part of a single log message, you may need to use additional clientͲside locking to prevent undesirable interleavingofoutputfrommultiplethreads.Iftwothreadsloggedmultilinestacktracestothesamestreamwithoneprintlncallperline,the resultswouldbeinterleavedunpredictably,andcouldeasilylooklikeonelargebutmeaninglessstacktrace.
Listing7.12.EncapsulatingNonstandardCancellationinaTaskwithNewtaskfor.
public interface CancellableTask<T> extends Callable<T> {
void cancel();
RunnableFuture<T> newTask();
}
@ThreadSafe
public class CancellingExecutor extends ThreadPoolExecutor {
...
protected<T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
if (callable instanceof CancellableTask)
return ((CancellableTask<T>) callable).newTask();
else
return super.newTaskFor(callable);
}
}
public abstract class SocketUsingTask<T>
implements CancellableTask<T> {
@GuardedBy("this") private Socket socket;
protected synchronized void setSocket(Socket s) { socket = s; }
public synchronized void cancel() {
try {
if (socket != null)
socket.close();
} catch (IOException ignored) { }
}
public RunnableFuture<T> newTask() {
return new FutureTask<T>(this) {
public boolean cancel(boolean mayInterruptIfRunning) {
try {
SocketUsingTask.this.cancel();
} finally {
return super.cancel(mayInterruptIfRunning);
}
}
};
}
}
LogWriter in Listing 7.13 shows a simple logging service in which the logging activity is moved to a separate logger thread.Insteadofhavingthethreadthatproducesthemessagewriteitdirectlytotheoutputstream,LogWriterhands itofftotheloggerthreadviaaBlockingQueueandtheloggerthreadwritesitout.ThisisamultipleͲproducer,singleͲ
consumerdesign:anyactivitycallinglogisactingasaproducer,andthebackgroundloggerthreadistheconsumer.If theloggerthreadfallsbehind,theBlockingQueueeventuallyblockstheproducersuntiltheloggerthreadcatchesup.
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 95
Listing7.13.ProducerǦConsumerLoggingServicewithNoShutdownSupport.
public class LogWriter {
private final BlockingQueue<String> queue;
private final LoggerThread logger;
public LogWriter(Writer writer) {
this.queue = new LinkedBlockingQueue<String>(CAPACITY);
this.logger = new LoggerThread(writer);
}
public void start() { logger.start(); }
public void log(String msg) throws InterruptedException {
queue.put(msg);
}
private class LoggerThread extends Thread {
private final PrintWriter writer;
...
public void run() {
try {
while (true)
writer.println(queue.take());
} catch(InterruptedException ignored) {
} finally {
writer.close();
}
}
}
}
ForaservicelikeLogWritertobeusefulinproduction,weneedawaytoterminatetheloggerthreadsoitdoesnot preventtheJVMfromshuttingdownnormally.Stoppingtheloggerthreadiseasyenough,sinceitrepeatedlycallstake, which is responsive to interruption; if the logger thread is modified to exit on catching InterruptedException, then interruptingtheloggerthreadstopstheservice.
However,simplymakingtheloggerthreadexitisnotaverysatifyingshutdownmechanism.Suchanabruptshutdown discards log messages that might be waiting to be written to the log, but, more importantly, threads blocked in log becausethequeueisfullwillneverbecomeunblocked.Cancellingaproducerconsumeractivityrequirescancellingboth theproducersandtheconsumers.Interruptingtheloggerthreaddealswiththeconsumer,butbecausetheproducersin thiscasearenotdedicatedthreads,cancellingthemisharder.
Another approach to shutting down LogWriter would be to set a "shutdown requested" flag to prevent further messagesfrombeingsubmitted,asshowninListing7.14.Theconsumercouldthendrainthequeueuponbeingnotified that shutdown has been requested, writing out any pending messages and unblocking any producers blocked in log.
However, this approach has race conditions that make it unreliable. The implementation of log is a checkͲthenͲact sequence: producers could observe that the service has not yet been shut down but still queue messages after the shutdown,againwiththeriskthattheproducermightgetblockedinlogandneverbecomeunblocked.Therearetricks thatreducethelikelihoodofthis(likehavingtheconsumerwaitseveralsecondsbeforedeclaringthequeuedrained), butthesedonotchangethefundamentalproblem,merelythelikelihoodthatitwillcauseafailure.
Listing7.14.UnreliableWaytoAddShutdownSupporttotheLoggingService.
public void log(String msg) throws InterruptedException {
if (!shutdownRequested)
queue.put(msg);
else
throw new IllegalStateException("logger is shut down");
}
ThewaytoprovidereliableshutdownforLogWriteristofixtheracecondition,whichmeansmakingthesubmissionof anewlogmessageatomic.Butwedon'twanttoholdalockwhiletryingtoenqueuethemessage,sinceputcouldblock.
Instead,wecanatomicallycheckforshutdownandconditionallyincrementacounterto"reserve"therighttosubmita message,asshowninLogServiceinListing7.15.
96 JavaConcurrencyInPractice
7.2.2. ExecutorServiceShutdown
InSection6.2.4,wesawthatExecutorServiceofferstwowaystoshutdown:gracefulshutdownwithshutdown,and abrupt shutdown with shutdownNow. In an abrupt shutdown, shutdownNow returns the list of tasks that had not yet startedafterattemptingtocancelallactivelyexecutingtasks.
Listing7.15.AddingReliableCancellationtoLogWriter.
public class LogService {
private final BlockingQueue<String> queue;
private final LoggerThread loggerThread;
private final PrintWriter writer;
@GuardedBy("this") private boolean isShutdown;
@GuardedBy("this") private int reservations;
public void start() { loggerThread.start(); }
public void stop() {
synchronized (this) { isShutdown = true; }
loggerThread.interrupt();
}
public void log(String msg) throws InterruptedException {
synchronized (this) {
if (isShutdown)
throw new IllegalStateException(...);
++reservations;
}
queue.put(msg);
}
private class LoggerThread extends Thread {
public void run() {
try {
while (true) {
try {
synchronized (this) {
if (isShutdown && reservations == 0)
break;
}
String msg = queue.take();
synchronized (this) { --reservations; }
writer.println(msg);
} catch (InterruptedException e) { /* retry */ }
}
} finally {
writer.close();
}
}
}
}
Thetwodifferentterminationoptionsofferatradeoffbetweensafetyandresponsiveness:abruptterminationisfaster but riskier because tasks may be interrupted in the middle of execution, and normal termination is slower but safer becausetheExecutorServicedoesnotshutdownuntilallqueuedtasksareprocessed.OtherthreadͲowningservices shouldconsiderprovidingasimilarchoiceofshutdownmodes.
Simple programs can get away with starting and shutting down a global ExecutorService from main. More sophisticated programs are likely to encapsulate an ExecutorService behind a higherͲlevel service that provides its ownlifecyclemethods,suchasthevariantofLogServiceinListing7.16thatdelegatestoanExecutorServiceinstead of managing its own threads. Encapsulating an ExecutorService extends the ownership chain from application to servicetothreadbyaddinganotherlink;eachmemberofthechainmanagesthelifecycleoftheservicesorthreadsit owns.
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 97
Listing7.16.LoggingServicethatUsesanExecutorService.
public class LogService {
private final ExecutorService exec = newSingleThreadExecutor();
...
public void start() { }
public void stop() throws InterruptedException {
try {
exec.shutdown();
exec.awaitTermination(TIMEOUT, UNIT);
} finally {
writer.close();
}
}
public void log(String msg) {
try {
exec.execute(new WriteTask(msg));
} catch (RejectedExecutionException ignored) { }
}
}
7.2.3.PoisonPills
AnotherwaytoconvinceaproducerͲconsumerservicetoshutdowniswithapoisonpill:arecognizableobjectplaced onthequeuethatmeans"whenyougetthis,stop."WithaFIFOqueue,poisonpillsensurethatconsumersfinishthe workontheirqueuebeforeshuttingdown,sinceanyworksubmittedpriortosubmittingthepoisonpillwillberetrieved before the pill; producers should not submit any work after putting a poison pill on the queue. IndexingService in Listings 7.17, 7.18, and 7.19 shows a singleͲproducer, singleͲconsumer version of the desktop search example from Listing5.8onpage91thatusesapoisonpilltoshutdowntheservice.
Listing7.17.ShutdownwithPoisonPill.
public class IndexingService {
private static final File POISON = new File("");
private final IndexerThread consumer = new IndexerThread();
private final CrawlerThread producer = new CrawlerThread();
private final BlockingQueue<File> queue;
private final FileFilter fileFilter;
private final File root;
class CrawlerThread extends Thread { /* Listing 7.18 */ }
class IndexerThread extends Thread { /* Listing 7.19 */ }
public void start() {
producer.start();
consumer.start();
}
public void stop() { producer.interrupt(); }
public void awaitTermination() throws InterruptedException {
consumer.join();
}
}
Poisonpillsworkonlywhenthenumberofproducersandconsumersisknown.TheapproachinIndexingServicecan beextendedtomultipleproducersbyhavingeachproducerplaceapillonthequeueandhavingtheconsumerstoponly whenitreceivesNproducerspills.ItcanbeextendedtomultipleconsumersbyhavingeachproducerplaceNconsumerspillson thequeue,thoughthiscangetunwieldywithlargenumbersofproducersandconsumers.Poisonpillsworkreliablyonly withunboundedqueues.
7.2.4.Example:AOneǦshotExecutionService
Ifamethodneedstoprocessabatchoftasksanddoesnotreturnuntilallthetasksarefinished,itcansimplifyservice lifecycle management by using a private Executor whose lifetime is bounded by that method. (The invokeAll and invokeAnymethodscanoftenbeusefulinsuchsituations.)
ThecheckMailmethodinListing7.20checksfornewmailinparallelonanumberofhosts.Itcreatesaprivateexecutor andsubmitsataskforeachhost:itthenshutsdowntheexecutorandwaitsfortermination,whichoccurswhenallthe mailͲcheckingtaskshavecompleted.[4]
[4]ThereasonanAtomicBooleanisusedinsteadofavolatile booleanisthatinordertoaccessthehasNewMailflagfromtheinner Runnable,itwouldhavetobefinal,whichwouldprecludemodifyingit.
98 JavaConcurrencyInPractice
Listing7.18.ProducerThreadforIndexingService.
public class CrawlerThread extends Thread {
public void run() {
try {
crawl(root);
} catch (InterruptedException e) { /* fall through */ }
finally {
while (true) {
try {
queue.put(POISON);
break;
} catch (InterruptedException e1) { /* retry */ }
}
}
}
private void crawl(File root) throws InterruptedException {
...
}
}
Listing7.19.ConsumerThreadforIndexingService.
public class IndexerThread extends Thread {
public void run() {
try {
while (true) {
File file = queue.take();
if (file == POISON)
break;
else
indexFile(file);
}
} catch (InterruptedException consumed) { }
}
}
Listing7.20.UsingaPrivateExecutorWhoseLifetimeisBoundedbyaMethodCall.
boolean checkMail(Set<String> hosts, long timeout, TimeUnit unit) throws InterruptedException {
ExecutorService exec = Executors.newCachedThreadPool();
final AtomicBoolean hasNewMail = new AtomicBoolean(false);
try {
for (final String host : hosts)
exec.execute(new Runnable() {
public void run() {
if (checkMail(host))
hasNewMail.set(true);
}
});
} finally {
exec.shutdown();
exec.awaitTermination(timeout, unit);
}
return hasNewMail.get();
}
7.2.5.LimitationsofShutdownnow
When an ExecutorService is shut down abruptly with shutdownNow, it attempts to cancel the tasks currently in progressandreturnsalistoftasksthatweresubmittedbutneverstartedsothattheycanbeloggedorsavedforlater processing.[5]
[5]TheRunnableobjectsreturnedbyshutdownNowmightnotbethesameobjectsthatweresubmittedtotheExecutorService:they mightbewrappedinstancesofthesubmittedtasks.
However,thereisnogeneralwaytofindoutwhichtasksstartedbutdidnotcomplete.Thismeansthatthereisnoway of knowing the state of the tasks in progress at shutdown time unless the tasks themselves perform some sort of checkpointing.Toknowwhichtaskshavenotcompleted,youneedtoknownotonlywhichtasksdidn'tstart,butalso whichtaskswereinprogresswhentheexecutorwasshutdown.[6]
[6] Unfortunately, there is no shutdown option in which tasks not yet started are returned to the caller but tasks in progress are allowed to complete;suchanoptionwouldeliminatethisuncertainintermediatestate.
TRackingExecutorinListing7.21showsatechniquefordeterminingwhichtaskswereinprogressatshutdowntime.By encapsulatinganExecutorServiceandinstrumentingexecute(andsimilarlysubmit,notshown)torememberwhich tasks were cancelled after shutdown, trackingExecutor can identify which tasks started but did not complete normally. After the executor terminates, getCancelledTasks returns the list of cancelled tasks. In order for this
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 99
techniquetowork,thetasksmustpreservethethread'sinterruptedstatuswhentheyreturn,whichwellbehavedtasks willdoanyway.
Listing7.21. ExecutorServicethatKeepsTrackofCancelledTasksAfterShutdown.
public class TrackingExecutor extends AbstractExecutorService {
private final ExecutorService exec;
private final Set<Runnable> tasksCancelledAtShutdown =
Collections.synchronizedSet(new HashSet<Runnable>());
...
public List<Runnable> getCancelledTasks() {
if (!exec.isTerminated())
throw new IllegalStateException(...);
return new ArrayList<Runnable>(tasksCancelledAtShutdown);
}
public void execute(final Runnable runnable) {
exec.execute(new Runnable() {
public void run() {
try {
runnable.run();
} finally {
if (isShutdown()
&& Thread.currentThread().isInterrupted())
tasksCancelledAtShutdown.add(runnable);
}
}
});
}
// delegate other ExecutorService methods to exec
}
WebCrawlerinListing7.22showsanapplicationoftrackingExecutor.Theworkofawebcrawlerisoftenunbounded, soifacrawlermustbeshutdownwemightwanttosaveitsstatesoitcanberestartedlater.CrawlTaskprovidesa getPagemethodthatidentifieswhatpageitisworkingon.Whenthecrawlerisshutdown,boththetasksthatdidnot startandthosethatwerecancelledarescannedandtheirURLsrecorded,sothatpageͲcrawlingtasksforthoseURLscan beaddedtothequeuewhenthecrawlerrestarts.
TRackingExecutorhasanunavoidableraceconditionthatcouldmakeityieldfalsepositives:tasksthatareidentifiedas cancelled but actually completed. This arises because the thread pool could be shut down between when the last instruction of the task executes and when the pool records the task as complete. This is not a problem if tasks are idempotent (if performing them twice has the same effect as performing them once), as they typically are in a web crawler. Otherwise, the application retrieving the cancelled tasks must be aware of this risk and be prepared to deal withfalsepositives.
100 JavaConcurrencyInPractice
Listing7.22.UsingTRackingExecutorServicetoSaveUnfinishedTasksforLaterExecution.
public abstract class WebCrawler {
private volatile TrackingExecutor exec;
@GuardedBy("this")
private final Set<URL> urlsToCrawl = new HashSet<URL>();
...
public synchronized void start() {
exec = new TrackingExecutor(
Executors.newCachedThreadPool());
for (URL url : urlsToCrawl) submitCrawlTask(url);
urlsToCrawl.clear();
}
public synchronized void stop() throws InterruptedException {
try {
saveUncrawled(exec.shutdownNow());
if (exec.awaitTermination(TIMEOUT, UNIT))
saveUncrawled(exec.getCancelledTasks());
} finally {
exec = null;
}
}
protected abstract List<URL> processPage(URL url);
private void saveUncrawled(List<Runnable> uncrawled) {
for (Runnable task : uncrawled)
urlsToCrawl.add(((CrawlTask) task).getPage());
}
private void submitCrawlTask(URL u) {
exec.execute(new CrawlTask(u));
}
private class CrawlTask implements Runnable {
private final URL url;
...
public void run() {
for (URL link : processPage(url)) {
if (Thread.currentThread().isInterrupted())
return;
submitCrawlTask(link);
}
}
public URL getPage() { return url; }
}
}
7.3.HandlingAbnormalThreadTermination
It is obvious when a singleͲthreaded console application terminates due to an uncaught exceptionthe program stops runningandproducesastacktracethatisverydifferentfromtypicalprogramoutput.Failureofathreadinaconcurrent applicationisnotalwayssoobvious.Thestacktracemaybeprintedontheconsole,butnoonemaybewatchingthe console.Also,whena threadfails,theapplication mayappear tocontinuetowork,soitsfailurecouldgounnoticed.
Fortunately,therearemeansofbothdetectingandpreventingthreadsfrom"leaking"fromanapplication.
TheleadingcauseofprematurethreaddeathisRuntimeException.Becausetheseexceptionsindicateaprogramming errororotherunrecoverableproblem,theyaregenerallynotcaught.Insteadtheypropagateallthewayupthestack,at whichpointthedefaultbehavioristoprintastacktraceontheconsoleandletthethreadterminate.
The consequences of abnormal thread death range from benign to disastrous, depending on the thread's role in the application.Losingathreadfromathreadpoolcanhaveperformanceconsequences,butanapplicationthatrunswell witha50Ͳthreadpoolwillprobablyrunfinewitha49Ͳthreadpooltoo.Butlosingtheeventdispatch threadina GUI application would be quite noticeable Ͳ the application would stop processing events and the GUI would freeze.
OutOfTime on 124 showed a serious consequence of thread leakage: the service represented by the Timer is permanentlyoutofcommission.
JustaboutanycodecanthrowaRuntimeException.Wheneveryoucallanothermethod,youaretakingaleapoffaith thatitwillreturnnormallyorthrowoneofthecheckedexceptionsitssignaturedeclares.Thelessfamiliaryouarewith thecodebeingcalled,themoreskepticalyoushouldbeaboutitsbehavior.
TaskͲprocessing threads such as the worker threads in a thread pool or the Swing event dispatch thread spend their whole life calling unknown code through an abstraction barrier like Runnable, and these threads should be very skepticalthatthecodetheycallwillbewellbehaved.ItwouldbeverybadifaserviceliketheSwingeventthreadfailed justbecausesomepoorlywritteneventhandlerthrewaNullPointerException.Accordingly,thesefacilitiesshouldcall taskswithinatry-catchblockthatcatchesuncheckedexceptions,orwithinatry-finallyblocktoensurethatifthe
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 101
threadexitsabnormallytheframeworkisinformedofthisandcantakecorrectiveaction.Thisisoneofthefewtimes whenyoumightwanttoconsidercatchingRuntimeExceptionwhenyouarecallingunknown,untrustedcodethrough anabstractionsuchasRunnable.[7]
[7]Thereissomecontroversyoverthesafetyofthistechnique;whenathreadthrowsanuncheckedexception,theentireapplicationmaypossibly becompromised.ButthealternativeͲshuttingdowntheentireapplicationͲisusuallynotpractical.
Listing7.23illustratesawaytostructureaworkerthreadwithinathreadpool.Ifataskthrowsanuncheckedexception, itallowsthethreadtodie,butnotbeforenotifyingtheframeworkthatthethreadhasdied.Theframeworkmaythen replacetheworkerthreadwithanewthread,ormaychoosenottobecausethethreadpoolisbeingshutdownorthere are already enough worker threads to meet current demand. ThreadPoolExecutor and Swing use this technique to ensurethatapoorlybehavedtaskdoesn'tpreventsubsequenttasksfromexecuting.Ifyouarewritingaworkerthread classthatexecutessubmittedtasks,orcallinguntrustedexternalcode(suchasdynamicallyloadedplugins),useoneof theseapproachestopreventapoorlywrittentaskorpluginfromtakingdownthethreadthathappenstocallit.
Listing7.23.TypicalThreadǦpoolWorkerThreadStructure.
public void run() {
Throwable thrown = null;
try {
while (!isInterrupted())
runTask(getTaskFromWorkQueue());
} catch (Throwable e) {
thrown = e;
} finally {
threadExited(this, thrown);
}
}
7.3.1.UncaughtExceptionHandlers
The previous section offered a proactive approach to the problem of unchecked exceptions. The Thread API also provides the UncaughtExceptionHandler facility, which lets you detect when a thread dies due to an uncaught exception. The two approaches are complementary: taken together, they provide defenseͲindepth against thread leakage.
When a thread exits due to an uncaught exception, the JVM reports this event to an applicationͲprovided UncaughtExceptionHandler (see Listing 7.24); if no handler exists, the default behavior is to print the stack trace to System.err.[8]
[8]BeforeJava5.0,theonlywaytocontroltheUncaughtExceptionHandlerwasbysubclassingThreadGroup.InJava5.0andlater,you cansetanUncaughtExceptionHandleronaperͲthreadbasiswithThread.setUncaughtExceptionHandler,andcanalsosetthe default UncaughtExceptionHandler with Thread.setDefaultUncaughtExceptionHandler. However, only one of these handlers is calledfirst the JVM looks for a perͲthread handler, then for a ThreadGroup handler. The default handler implementation in ThreadGroupdelegatestoitsparentthreadgroup,andsoonupthechainuntiloneoftheThreadGrouphandlersdealswiththeuncaught exceptionoritbubblesuptothetoplevelthreadgroup.ThetopͲlevelthreadgrouphandlerdelegatestothedefaultsystemhandler(ifoneexists; thedefaultisnone)andotherwiseprintsthestacktracetotheconsole.
Listing7.24. UncaughtExceptionHandlerInterface.
public interface UncaughtExceptionHandler {
void uncaughtException(Thread t, Throwable e);
}
What the handler should do with an uncaught exception depends on your qualityͲofͲservice requirements. The most commonresponseistowriteanerrormessageandstacktracetotheapplicationlog,asshowninListing7.25.Handlers canalsotakemoredirectaction,suchastryingtorestartthethread,shuttingdowntheapplication,paginganoperator, orothercorrectiveordiagnosticaction.
Listing7.25. UncaughtExceptionHandlerthatLogstheException.
public class UEHLogger implements Thread.UncaughtExceptionHandler {
public void uncaughtException(Thread t, Throwable e) {
Logger logger = Logger.getAnonymousLogger();
logger.log(Level.SEVERE,
"Thread terminated with exception: " + t.getName(),
e);
}
}
InlongͲrunningapplications,alwaysuseuncaughtexceptionhandlersforallthreadsthatatleastlogtheexception.
102 JavaConcurrencyInPractice
To set an UncaughtExceptionHandler for pool threads, provide a ThreadFactory to the ThreadPoolExecutor constructor.(Aswithallthreadmanipulation,onlythethread'sownershouldchangeitsUncaughtExceptionHandler.) Thestandardthreadpoolsallowanuncaughttaskexceptiontoterminatethepoolthread,butuseatry-finallyblock tobenotifiedwhenthishappenssothethreadcanbereplaced.Withoutanuncaughtexceptionhandlerorotherfailure notificationmechanism,taskscanappeartofailsilently,whichcanbeveryconfusing.Ifyouwanttobenotifiedwhena task fails due to an exception so that you can take some taskͲspecific recovery action, either wrap the task with a RunnableorCallablethatcatchestheexceptionoroverridetheafterExecutehookinThreadPoolExecutor.
Somewhat confusingly, exceptions thrown from tasks make it to the uncaught exception handler only for tasks submittedwithexecute;fortaskssubmittedwithsubmit,anythrownexception,checkedornot,isconsideredtobe part of the task's return status. If a task submitted with submit terminates with an exception, it is rethrown by Future.get,wrappedinanExecutionException.
7.4.JVMShutdown
TheJVMcanshutdownineitheranorderlyorabruptmanner.Anorderlyshutdownisinitiatedwhenthelast"normal"
(nonͲdaemon)threadterminates,someonecallsSystem.exit,orbyotherplatformͲspecificmeans(suchassendinga SIGINTorhittingCtrl-C).WhilethisisthestandardandpreferredwayfortheJVMtoshutdown,itcanalsobeshut downabruptlybycallingRuntime.haltorbykillingtheJVMprocessthroughtheoperatingsystem(suchassendinga SIGKILL).
7.4.1.ShutdownHooks
Inanorderlyshutdown,theJVMfirststartsallregisteredshutdownhooks.Shutdownhooksareunstartedthreadsthat areregisteredwithRuntime.addShutdownHook.TheJVMmakesnoguaranteesontheorderinwhichshutdownhooks arestarted.Ifanyapplicationthreads(daemonornondaemon)arestillrunningatshutdowntime,theycontinuetorun concurrently with the shutdown process. When all shutdown hooks have completed, the JVM may choose to run finalizers if runFinalizersOnExit is true, and then halts. The JVM makes no attempt to stop or interrupt any applicationthreadsthatarestillrunningatshutdowntime;theyareabruptlyterminatedwhentheJVMeventuallyhalts.
Iftheshutdownhooksorfinalizersdon'tcomplete,thentheorderlyshutdownprocess"hangs"andtheJVMmustbe shutdownabruptly.Inanabruptshutdown,theJVMisnotrequiredtodoanythingotherthanhalttheJVM;shutdown hookswillnotrun.
Shutdown hooks should be threadͲsafe: they must use synchronization when accessing shared data and should be careful to avoid deadlock, just like any other concurrent code. Further, they should not make assumptions about the stateoftheapplication(suchaswhetherotherserviceshaveshutdownalreadyorallnormalthreadshavecompleted) oraboutwhytheJVMisshuttingdown,andmustthereforebecodedextremelydefensively.Finally,theyshouldexitas quicklyaspossible,sincetheirexistencedelaysJVMterminationatatimewhentheusermaybeexpectingtheJVMto terminatequickly.
Shutdown hooks can be used for service or application cleanup, such as deleting temporary files or cleaning up resources that are not automatically cleaned up by the OS. Listing 7.26 shows how LogService in Listing 7.16 could registerashutdownhookfromitsstartmethodtoensurethelogfileisclosedonexit.
Becauseshutdownhooks allrun concurrently, closingthelogfilecouldcausetroubleforothershutdownhookswho wanttousethelogger.Toavoidthisproblem,shutdownhooksshouldnotrelyonservicesthatcanbeshutdownbythe application or other shutdown hooks. One way to accomplish this is to use a single shutdown hook for all services, rather than one for each service, and have it call a series of shutdown actions. This ensures that shutdown actions executesequentiallyinasinglethread,thusavoidingthepossibilityofraceconditionsordeadlockbetweenshutdown actions.Thistechniquecanbeusedwhetherornotyouuseshutdownhooks;executingshutdownactionssequentially ratherthanconcurrentlyeliminatesmanypotentialsourcesoffailure.Inapplicationsthatmaintainexplicitdependency informationamongservices,thistechniquecanalsoensurethatshutdownactionsareperformedintherightorder.
5BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown 103
Listing7.26.RegisteringaShutdownHooktoStoptheLoggingService.
public void start() {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
try { LogService.this.stop(); }
catch (InterruptedException ignored) {}
}
});
}
7.4.2.DaemonThreads
Sometimesyouwanttocreateathreadthatperformssomehelperfunctionbutyoudon'twanttheexistenceofthis threadtopreventtheJVMfromshuttingdown.Thisiswhatdaemonthreadsarefor.
Threads are divided into two types: normal threads and daemon threads. When the JVM starts up, all the threads it creates(suchasgarbagecollectorandotherhousekeepingthreads)aredaemonthreads,exceptthemainthread.When anewthreadiscreated,itinheritsthedaemonstatusofthethreadthatcreatedit,sobydefaultanythreadscreatedby themainthreadarealsonormalthreads.
Normal threads and daemon threads differ only in what happens when they exit. When a thread exits, the JVM
performsaninventoryofrunningthreads,andiftheonlythreadsthatareleftaredaemonthreads,itinitiatesanorderly shutdown. When the JVM halts, any remaining daemon threads are abandoned Ͳ finally blocks are not executed, stacksarenotunwoundͲtheJVMjustexits.
Daemon threads should be used sparingly Ͳ few processing activities can be safely abandoned at any time with no cleanup. In particular, it is dangerous to use daemon threads for tasks that might perform any sort of I/O. Daemon threadsarebestsavedfor"housekeeping"tasks,suchasabackgroundthreadthatperiodicallyremovesexpiredentries fromaninͲmemorycache.
Daemonthreadsarenotagoodsubstituteforproperlymanagingthelifecycleofserviceswithinanapplication.
7.4.3.Finalizers
The garbage collector does a good job of reclaiming memory resources when they are no longer needed, but some resources,suchasfileorsockethandles,mustbeexplicitlyreturnedtotheoperatingsystemwhennolongerneeded.To assist in this, the garbage collector treats objects that have a nontrivial finalize method specially: after they are reclaimedbythecollector,finalizeiscalledsothatpersistentresourcescanbereleased.
SincefinalizerscanruninathreadmanagedbytheJVM,anystateaccessedbyafinalizerwillbeaccessedbymorethan onethreadandthereforemustbeaccessedwithsynchronization.Finalizersoffernoguaranteesonwhenorevenifthey run, and they impose a significant performance cost on objects with nontrivial finalizers. They are also extremely difficulttowritecorrectly.[9]Inmostcases,thecombinationoffinallyblocksandexplicitclosemethodsdoesabetter jobofresourcemanagementthanfinalizers;thesoleexceptioniswhenyouneedtomanageobjectsthatholdresources acquiredby nativemethods.Forthesereasonsandothers,workhardtoavoidwritingor usingclasseswithfinalizers (otherthantheplatformlibraryclasses)[EJItem6].
[9]See(Boehm,2005)forsomeofthechallengesinvolvedinwritingfinalizers.
Avoidfinalizers.
Summary
EndͲofͲlifecycle issues for tasks, threads, services, and applications can add complexity to their design and implementation. Java does not provide a preemptive mechanism for cancelling activities or terminating threads.
Instead,itprovidesacooperativeinterruptionmechanismthatcanbeusedtofacilitatecancellation,butitisuptoyou to construct protocols for cancellation and use them consistently. Using FutureTask and the Executor framework simplifiesbuildingcancellabletasksandservices.
104 JavaConcurrencyInPractice
Chapter8.ApplyingThreadPools
Chapter 6 introduced the task execution framework, which simplifies management of task and thread lifecycles and providesasimpleandflexiblemeansfordecouplingtasksubmissionfromexecutionpolicy.Chapter7coveredsomeof the messy details of service lifecycle that arise from using the task execution framework in real applications. This chapterlooksatadvancedoptionsforconfiguringandtuningthreadpools,describeshazardstowatchforwhenusing thetaskexecutionframework,andofferssomemoreadvanced
8.1.ImplicitCouplingsBetweenTasksandExecutionPolicies
WeclaimedearlierthattheExecutorframeworkdecouplestasksubmissionfromtaskexecution.Likemanyattemptsat decoupling complex processes, this was a bit of an overstatement. While the Executor framework offers substantial flexibilityinspecifyingandmodifyingexecutionpolicies,notalltasksarecompatiblewithallexecutionpolicies.Typesof tasksthatrequirespecificexecutionpoliciesinclude:
Dependenttasks.Themostwellbehavedtasksareindependent:thosethatdonotdependonthetiming,results,orside effects of other tasks. When executing independent tasks in a thread pool, you can freely vary the pool size and configurationwithoutaffectinganythingbutperformance.Ontheotherhand,whenyousubmittasksthatdependon othertaskstoathreadpool,youimplicitlycreateconstraintsontheexecutionpolicythatmustbecarefullymanagedto avoidlivenessproblems(seeSection8.1.1).
Tasksthatexploitthreadconfinement.SingleͲthreadedexecutorsmakestrongerpromisesaboutconcurrencythando arbitrarythreadpools.Theyguaranteethattasksarenotexecutedconcurrently,whichallowsyoutorelaxthethread safetyoftaskcode.Objectscanbeconfinedtothetaskthread,thusenablingtasksdesignedtoruninthatthreadto access those objects without synchronization, even if those resources are not threadͲsafe. This forms an implicit coupling between the task and the execution policy Ͳ the tasks require their executor to be singleͲthreaded.[1] In this case,ifyouchangedtheExecutorfromasingleͲthreadedonetoathreadpool,threadsafetycouldbelost.
[1] The requirement is not quite this strong; it would be enough to ensure only that tasks not execute concurrently and provide enough synchronizationsothatthememoryeffectsofonetaskareguaranteedtobevisibletothenexttaskͲwhichispreciselytheguaranteeofferedby newSingle-ThreadExecutor.
ResponseͲtimeͲsensitive tasks. GUI applications are sensitive to response time: users are annoyed at long delays between a button click and the corresponding visual feedback. Submitting a longͲrunning task to a singleͲthreaded executor, or submitting several longͲrunning tasks to a thread pool with a small number of threads, may impair the responsivenessoftheservicemanagedbythatExecutor.
TasksthatuseThreadLocal.ThreadLocalallowseachthreadtohaveitsownprivate"version"ofavariable.However, executorsarefreetoreusethreadsastheyseefit.ThestandardExecutorimplementationsmayreapidlethreadswhen demand is low and add new ones when demand is high, and also replace a worker thread with a fresh one if an uncheckedexceptionisthrownfromatask. ThreadLocal makessensetouseinpoolthreadsonlyifthethreadͲlocal valuehasalifetimethatisboundedbythatofatask;Thread-Localshouldnotbeusedinpoolthreadstocommunicate valuesbetweentasks.
Thread pools work best when tasks are homogeneous and independent. Mixing longͲrunning and shortͲrunning tasks risks"clogging"thepoolunlessitisverylarge;submitting tasksthat dependonothertasksrisksdeadlockunlessthe poolisunbounded.Fortunately,requestsintypicalnetworkͲbasedserverapplicationsͲwebservers,mailservers,file serversͲusuallymeettheseguidelines.
Sometaskshavecharacteristicsthatrequireorprecludeaspecificexecutionpolicy.Tasksthatdependonothertasks require that the thread pool be large enough that tasks are never queued or rejected; tasks that exploit thread confinementrequiresequentialexecution.Documenttheserequirementssothatfuturemaintainersdonotundermine safetyorlivenessbysubstitutinganincompatibleexecutionpolicy.
8.1.1.ThreadStarvationDeadlock
Iftasksthatdependonothertasksexecuteinathreadpool,theycandeadlock.InasingleͲthreadedexecutor,atask thatsubmitsanothertasktothesameexecutorandwaitsforitsresultwillalwaysdeadlock.Thesecondtasksitsonthe work queue until the first task completes, but the first will not complete because it is waiting for the result of the secondtask.Thesamethingcanhappeninlargerthreadpoolsifallthreadsareexecutingtasksthatareblockedwaiting forothertasksstillontheworkqueue.Thisiscalledthreadstarvationdeadlock,andcanoccurwheneverapooltask initiates an unbounded blocking wait for some resource or condition that can succeed only through the action of
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 105
anotherpooltask,suchaswaitingforthereturnvalueorsideeffectofanothertask,unlessyoucanguaranteethatthe poolislargeenough.
ThreadDeadlockinListing8.1illustratesthreadstarvationdeadlock.Render-PageTasksubmitstwoadditionaltasksto theExecutortofetchthepageheaderandfooter,rendersthepagebody,waitsfortheresultsoftheheaderandfooter tasks, and then combines the header, body, and footer into the finished page. With a singleͲthreaded executor, ThreadDeadlockwillalwaysdeadlock.Similarly,taskscoordinatingamongstthemselveswithabarriercouldalsocause threadstarvationdeadlockifthepoolisnotbigenough.
WheneveryousubmittoanExecutortasksthatarenotindependent,beawareofthepossibilityofthreadstarvation deadlock, and document any pool sizing or configuration constraints in the code or configuration file where the Executorisconfigured.
Inadditiontoanyexplicitboundsonthesizeofathreadpool,theremayalsobeimplicitlimitsbecauseofconstraintson otherresources.IfyourapplicationusesaJDBCconnectionpoolwithtenconnectionsandeachtaskneedsadatabase connection, it is as if your thread pool only has ten threads because tasks in excess of ten will block waiting for a connection.
Listing8.1.TaskthatDeadlocksinaSingleǦthreadedExecutor.
public class ThreadDeadlock {
ExecutorService exec = Executors.newSingleThreadExecutor();
public class RenderPageTask implements Callable<String> {
public String call() throws Exception {
Future<String> header, footer;
header = exec.submit(new LoadFileTask("header.html"));
footer = exec.submit(new LoadFileTask("footer.html"));
String page = renderBody();
// Will deadlock -- task waiting for result of subtask
return header.get() + page + footer.get();
}
}
}
8.1.2.LongǦrunningTasks
Threadpoolscanhaveresponsivenessproblemsiftaskscanblockforextendedperiodsoftime,evenifdeadlockisnota possibility.AthreadpoolcanbecomecloggedwithlongͲrunningtasks,increasingtheservicetimeevenforshorttasks.If the pool size is too small relative to the expected steadyͲstate number of longrunning tasks, eventually all the pool threadswillberunninglongͲrunningtasksandresponsivenesswillsuffer.
OnetechniquethatcanmitigatetheilleffectsoflongͲrunningtasksisfortaskstousetimedresourcewaitsinsteadof unbounded waits. Most blocking methods in the plaform libraries come in both untimed and timed versions, such as Thread.join,BlockingQueue.put,CountDownLatch.await,andSelector.select.Ifthewaittimesout,youcanmark thetaskasfailedandabortitorrequeueitforexecutionlater.Thisguaranteesthateachtaskeventuallymakesprogress towards either successful or failed completion, freeing up threads for tasks that might complete more quickly. If a threadpoolisfrequentlyfullofblockedtasks,thismayalsobeasignthatthepool 8.2.SizingThreadPools
The ideal size for a thread pool depends on the types of tasks that will be submitted and the characteristics of the deployment system. Thread pool sizes should rarely be hardͲcoded; instead pool sizes should be provided by a configurationmechanismorcomputeddynamicallybyconsultingRuntime.availableProcessors.
Sizing thread pools is not an exact science, but fortunately you need only avoid the extremes of "too big" and "too small". If a thread pool is too big, then threads compete for scarce CPU and memory resources, resulting in higher memoryusageandpossibleresourceexhaustion.Ifitistoosmall,throughputsuffersasprocessorsgounuseddespite availablework.
Tosizeathreadpoolproperly,youneedtounderstandyour computing environment,yourresource budget,and the natureofyourtasks.Howmanyprocessorsdoesthedeploymentsystemhave?Howmuchmemory?Dotasksperform
106 JavaConcurrencyInPractice
mostlycomputation,I/O,orsomecombination?Dotheyrequireascarceresource,suchasaJDBCconnection?Ifyou have different categories of tasks with very different behaviors, consider using multiple thread pools so each can be tunedaccordingtoitsworkload.
ForcomputeͲintensivetasks,anNcpuͲprocessorsystemusuallyachievesoptimumutilizationwithathreadpoolofNcpu
+1 threads. (Even computeͲintensive threads occasionally take a page fault or pause for some other reason, so an
"extra"runnablethreadpreventsCPUcyclesfromgoingunusedwhenthishappens.)FortasksthatalsoincludeI/Oor otherblockingoperations,youwantalargerpool,sincenotallofthethreadswillbeschedulableatalltimes.Inorderto sizethepoolproperly,youmustestimatetheratioofwaitingtimetocomputetimeforyourtasks;thisestimateneed notbepreciseandcanbeobtainedthroughproͲfilingorinstrumentation.Alternatively,thesizeofthethreadpoolcan betunedbyrunningtheapplicationusingseveraldifferentpoolsizesunderabenchmarkloadandobservingthelevelof CPUutilization.
Giventhesedefinitions:
ܰ௨ = number of CPUs
ܷ௨ = target CPU utilization, Ͳ ܷ௨ ͳ
ܹ = ratio of wait time to compute time
ܥ
Theoptimalpoolsizeforkeepingtheprocessorsatthedesiredutilizationis:
ܹ
ܰ௧ௗ௦ ൌ ܰ௨ כ ܷ௨ כ ൬ͳ ൰
ܥ
YoucandeterminethenumberofCPUsusingRuntime:
int N_CPUS = Runtime.getRuntime().availableProcessors();
Ofcourse,CPUcyclesarenottheonlyresourceyoumightwanttomanageusingthreadpools.Otherresourcesthatcan contribute to sizing constraints are memory, file handles, socket handles, and database connections. Calculating pool size constraints for these types of resources is easier: just add up how much of that resource each task requires and dividethatintothetotalquantityavailable.Theresultwillbeanupperboundonthepoolsize.
Whentasksrequireapooledresourcesuchasdatabaseconnections,threadpoolsizeandresourcepoolsizeaffecteach other. If each task requires a connection, the effective size of the thread pool is limited by the connection pool size.
Similarly,whentheonlyconsumersofconnectionsarepooltasks,theeffectivesizeoftheconnectionpoolislimitedby thethreadpoolsize.
8.3.ConfiguringThreadPoolExecutor
ThreadPoolExecutor provides the base implementation for the executors returned by the newCachedThreadPool, newFixedThreadPool, and newScheduled-ThreadExecutor factories in Executors. ThreadPoolExecutor is a flexible, robustpoolimplementationthatallowsavarietyofcustomizations.
If the default execution policy does not meet your needs, you can instantiate a ThreadPoolExecutor through its constructorandcustomizeitasyouseefit;youcanconsultthesourcecodeforExecutorstoseetheexecutionpolicies forthedefaultconfigurationsandusethemasastartingpoint.ThreadPoolExecutorhasseveralconstructors,themost generalofwhichisshowninListing8.2.
8.3.1.ThreadCreationandTeardown
Thecorepoolsize,maximumpoolsize,andkeepͲalivetimegovernthreadcreationandteardown.Thecoresizeisthe targetsize;theimplementationattemptstomaintainthepoolatthissizeevenwhentherearenotaskstoexecute,[2]
andwillnotcreatemorethreadsthanthisunlesstheworkqueueisfull.[3]Themaximumpoolsizeistheupperboundon howmanypoolthreadscanbeactiveatonce.AthreadthathasbeenidleforlongerthanthekeepͲalivetimebecomesa candidateforreapingandcanbeterminatedifthecurrentpoolsizeexceedsthecoresize.
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 107
[2]WhenaThreadPoolExecutorisinitiallycreated,thecorethreadsarenotstartedimmediatelybutinsteadastasksaresubmitted,unless youcallprestartAllCoreThreads.
[3]Developersaresometimestemptedtosetthecoresizetozerosothattheworkerthreadswilleventuallybetorndownandthereforewon't preventtheJVMfromexiting,butthiscancausesomestrangeͲseemingbehaviorinthreadpoolsthatdon'tuseaSynchronousQueuefortheir workqueue(asnewCachedThreadPooldoes).Ifthepoolisalreadyatthecoresize,ThreadPoolExecutorcreatesanewthreadonlyif theworkqueueisfull.Sotaskssubmittedtoathreadpoolwithaworkqueuethathasanycapacityandacoresizeofzerowillnotexecuteuntil thequeuefillsup,whichisusuallynotwhatisdesired.InJava6,allowCoreThreadTimeOutallowsyoutorequestthatallpoolthreadsbe abletotimeout;enablethisfeaturewithacoresizeofzeroifyouwantaboundedthreadpoolwithaboundedworkqueuebutstillhaveallthe threadstorndownwhenthereisnoworktodo.
Listing8.2.GeneralConstructorforThreadPoolExecutor.
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) { ... }
BytuningthecorepoolsizeandkeepͲalivetimes,youcanencouragethepooltoreclaimresourcesusedbyotherwise idlethreads,makingthemavailableformoreusefulwork.(Likeeverythingelse,thisisatradeoff:reapingidlethreads incursadditionallatencyduetothreadcreationifthreadsmustlaterbecreatedwhendemandincreases.) ThenewFixedThreadPoolfactorysetsboththecorepoolsizeandthemaximumpoolsizetotherequestedpoolsize, creating the effect of infinite timeout; the newCachedThreadPool factory sets the maximum pool size to Integer.MAX_VALUE and the core pool size to zero with a timeout of one minute, creating the effect of an infinitely expandable thread pool that will contract again when demand decreases. Other combinations are possible using the explicitThreadPool-Executorconstructor.
8.3.2.ManagingQueuedTasks
Boundedthreadpoolslimitthenumberoftasksthatcanbeexecutedconcurrently.(ThesingleͲthreadedexecutorsarea notablespecialcase:theyguaranteethatnotaskswillexecuteconcurrently,offeringthepossibilityofachievingthread safetythroughthreadconfinement.)
WesawinSection6.1.2howunboundedthreadcreationcouldleadtoinstability,andaddressedthisproblembyusinga fixedͲsizedthreadpoolinsteadofcreatinganewthreadforeveryrequest.However,thisisonlyapartialsolution;itis stillpossiblefortheapplicationtorunoutofresourcesunderheavyload,justharder.Ifthearrivalratefornewrequests exceedstherateatwhichtheycanbehandled,requestswillstillqueueup.Withathreadpool,theywaitinaqueueof RunnablesmanagedbytheExecutorinsteadofqueueingupasthreadscontendingfortheCPU.Representingawaiting taskwithaRunnableandalistnodeiscertainlyalotcheaperthanwithathread,buttheriskofresourceexhaustionstill remainsifclientscanthrowrequestsattheserverfasterthanitcanhandlethem.
Requests often arrive in bursts even when the average request rate is fairly stable. Queues can help smooth out transientburstsoftasks,butiftaskscontinuetoarrivetooquicklyyouwilleventuallyhavetothrottlethearrivalrateto avoidrunningoutofmemory.[4]Evenbeforeyourunoutofmemory,responsetimewillgetprogressivelyworseasthe taskqueuegrows.
[4]Thisisanalogoustoflowcontrolincommunicationsnetworks:youmaybewillingtobufferacertainamountofdata,buteventuallyyouneed tofindawaytogettheothersidetostopsendingyoudata,orthrowtheexcessdataonthefloorandhopethesenderretransmitsitwhenyou're notsobusy.
ThreadPoolExecutor allows you to supply a BlockingQueue to hold tasks awaiting execution. There are three basic approaches to task queuing: unbounded queue, bounded queue, and synchronous handoff. The choice of queue interactswithotherconfigurationparameterssuchaspoolsize.
ThedefaultfornewFixedThreadPoolandnewSingleThreadExecutoristouseanunboundedLinkedBlockingQueue.
Taskswillqueueupifallworkerthreadsarebusy,butthequeuecouldgrowwithoutboundifthetaskskeeparriving fasterthantheycanbeexecuted.
A more stable resource management strategy is to use a bounded queue, such as an ArrayBlockingQueue or a boundedLinkedBlockingQueueorPriority-BlockingQueue.Boundedqueueshelppreventresourceexhaustionbut introducethequestionofwhattodowithnewtaskswhenthequeueisfull.(Thereareanumberofpossiblesaturation policiesforaddressingthisproblem;seeSection8.3.3.)Withaboundedworkqueue,thequeuesizeandpoolsizemust betuned together. Alargequeuecoupledwithasmallpoolcanhelpreducememoryusage,CPUusage,andcontext switching,atthecostofpotentiallyconstrainingthroughput.
108 JavaConcurrencyInPractice
For very large or unbounded pools, you can also bypass queuing entirely and instead hand off tasks directly from producers to worker threads using a SynchronousQueue. A SynchronousQueue is not really a queue at all, but a mechanism for managing handoffs between threads. In order to put an element on a SynchronousQueue, another threadmustalreadybewaitingtoacceptthehandoff.Ifnothreadiswaitingbutthecurrentpoolsizeislessthanthe maximum, Thread-PoolExecutor creates a new thread; otherwise the task is rejected according to the saturation policy.Usingadirecthandoffismoreefficientbecausethetaskcanbehandedrighttothethreadthatwillexecuteit, ratherthanfirstplacingitonaqueueandthenhavingtheworkerthreadfetchitfromthequeue.SynchronousQueueis a practical choice only if the pool is unbounded or if rejecting excess tasks is acceptable. The newCachedThreadPool factoryusesaSynchronousQueue.
UsingaFIFOqueuelikeLinkedBlockingQueueorArrayBlockingQueuecausestaskstobestartedintheorderinwhich theyarrived.Formorecontrolovertaskexecutionorder,youcanuseaPriorityBlockingQueue,whichorderstasks accordingtopriority.Prioritycanbedefinedbynaturalorder(iftasksimplementComparable)orbyaComparator.
The newCachedThreadPool factory is a good default choice for an Executor, providing better queuing performance thanafixedthreadpool.[5]Afixedsizethreadpoolisagoodchoicewhenyouneedtolimitthenumberofconcurrent tasks for resourceͲmanagement purposes, as in a server application that accepts requests from network clients and wouldotherwisebevulnerabletooverload.
[5]ThisperformancedifferencecomesfromtheuseofSynchronousQueueinsteadofLinkedBlocking-Queue.SynchronousQueue wasreplacedinJava6withanewnonͲblockingalgorithmthatimprovedthroughputinExecutorbenchmarksbyafactorofthreeovertheJava 5.0SynchronousQueueimplementation(Schereretal.,2006).
Boundingeitherthethreadpoolortheworkqueueissuitableonlywhentasksareindependent.Withtasksthatdepend onothertasks,boundedthreadpoolsorqueuescancausethreadstarvationdeadlock;instead,useanunboundedpool configurationlikenewCachedThreadPool.[6]
[6] An alternative configuration for tasks that submit other tasks and wait for their results is to use a bounded thread pool, a SynchronousQueueastheworkqueue,andthecallerͲrunssaturationpolicy.
8.3.3.SaturationPolicies
When a bounded work queue fills up, the saturation policy comes into play. The saturation policy for a ThreadPoolExecutor can be modified by calling setRejectedExecutionHandler. (The saturation policy is also used when a task is submitted to an Executor that has been shut down.) Several implementations of RejectedExecutionHandler are provided, each implementing a different saturation policy: AbortPolicy, CallerRunsPolicy,DiscardPolicy,andDiscardOldestPolicy.
Thedefaultpolicy,abort,causesexecutetothrowtheuncheckedRejected-ExecutionException;thecallercancatch this exception and implement its own overflow handling as it sees fit. The discard policy silently discards the newly submittedtaskifitcannotbequeuedforexecution;thediscardͲoldestpolicydiscardsthetaskthatwouldotherwisebe executed next and tries to resubmit the new task. (If the work queue is a priority queue, this discards the highestͲ
priorityelement,sothecombinationofadiscardͲoldestsaturationpolicyandapriorityqueueisnotagoodone.) ThecallerͲrunspolicyimplementsaformofthrottlingthatneitherdiscardstasksnorthrowsanexception,butinstead tries to slow down the flow of new tasks by pushing some of the work back to the caller. It executes the newly submittedtasknotinapoolthread,butinthethreadthatcallsexecute.IfwemodifiedourWebServerexampletouse aboundedqueueandthecallerͲrunspolicy,afterallthepoolthreadswereoccupiedandtheworkqueuefilledupthe nexttaskwouldbeexecutedinthemainthreadduringthecalltoexecute.Sincethiswouldprobablytakesometime, themainthreadcannotsubmitanymoretasksforatleastalittlewhile,givingtheworkerthreadssometimetocatch uponthebacklog.Themainthreadwouldalsonotbecallingacceptduringthistime,soincomingrequestswillqueue upintheTCPlayerinsteadofintheapplication.Iftheoverloadpersisted,eventuallytheTCPlayerwoulddecideithas queued enough connection requests and begin discarding connection requests as well. As the server becomes overloaded,theoverloadisgraduallypushedoutwardͲfromthepoolthreadstotheworkqueuetotheapplicationto theTCPlayer,andeventuallytotheclientͲenablingmoregracefuldegradationunderload.
Choosing a saturation policy or making other changes to the execution policy can be done when the Executor is created.Listing8.3illustratescreatingafixedsizethreadpoolwiththecallerͲrunssaturationpolicy.
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 109
Listing8.3.CreatingaFixedǦsizedThreadPoolwithaBoundedQueueandtheCallerǦrunsSaturationPolicy.
ThreadPoolExecutor executor
= new ThreadPoolExecutor(N_THREADS, N_THREADS,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(CAPACITY));
executor.setRejectedExecutionHandler(
new ThreadPoolExecutor.CallerRunsPolicy());
Thereisnopredefinedsaturationpolicytomakeexecuteblockwhentheworkqueueisfull.However,thesameeffect canbeaccomplishedbyusingaSemaphoretoboundthetaskinjectionrate,asshowninBoundedExecutorinListing8.4.
Insuchanapproach,useanunboundedqueue(there'snoreasontoboundboththequeuesizeandtheinjectionrate) andsettheboundonthesemaphoretobeequaltothepoolsizeplusthenumberofqueuedtasksyouwanttoallow, sincethesemaphoreisboundingthenumberoftasksbothcurrentlyexecutingandawaitingexecution.
8.3.4.ThreadFactories
Whenever a thread pool needs to create a thread, it does so through a thread factory (see Listing 8.5). The default threadfactorycreatesanew,nondaemonthreadwithnospecialconfiguration.Specifyingathreadfactoryallowsyouto customizetheconfigurationofpoolthreads.ThreadFactoryhasasinglemethod,newThread,thatiscalledwhenevera threadpoolneedstocreateanewthread.
There are a number of reasons to use a custom thread factory. You might want to specify an UncaughtExceptionHandler for pool threads, or instantiate an instance of a custom Thread class, such as one that performsdebuglogging.Youmightwanttomodifythepriority(generallynotaverygoodidea;seeSection10.3.1)or setthedaemonstatus(again,notallthatgoodanidea;seeSection7.4.2)ofpoolthreads.Ormaybeyoujustwantto givepoolthreadsmoremeaningfulnamestosimplifyinterpretingthreaddumpsanderrorlogs.
Listing8.4.UsingaSemaphoretoThrottleTaskSubmission.
@ThreadSafe
public class BoundedExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command)
throws InterruptedException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
}
}
}
Listing8.5. ThreadFactoryInterface.
public interface ThreadFactory {
Thread newThread(Runnable r);
}
MyThreadFactoryinListing8.6illustratesacustomthreadfactory.ItinstantiatesanewMyAppThread,passingapoolͲ
specificnametotheconstructorsothatthreadsfromeachpoolcanbedistinguishedinthreaddumpsanderrorlogs.
My-AppThread can also be used elsewhere in the application so that all threads can take advantage of its debugging features.
110 JavaConcurrencyInPractice
Listing8.6.CustomThreadFactory.
public class MyThreadFactory implements ThreadFactory {
private final String poolName;
public MyThreadFactory(String poolName) {
this.poolName = poolName;
}
public Thread newThread(Runnable runnable) {
return new MyAppThread(runnable, poolName);
}
}
TheinterestingcustomizationtakesplaceinMyAppThread,showninListing8.7,whichletsyouprovideathreadname, sets a custom UncaughtException-Handler that writes a message to a Logger, maintains statistics on how many threadshavebeencreatedanddestroyed,andoptionallywritesadebugmessagetothelogwhenathreadiscreatedor terminates.
Ifyourapplicationtakesadvantageofsecuritypoliciestograntpermissionstoparticularcodebases,youmaywantto use the privilegedThreadFactory factory method in Executors to construct your thread factory. It creates pool threadsthathavethesamepermissions,AccessControlContext,andcontextClassLoaderasthethreadcreatingthe privilegedThreadFactory. Otherwise, threads created by the thread pool inherit permissions from whatever client happens to be calling execute or submit at the time a new thread is needed, which could cause confusing securityͲ
relatedexceptions.
8.3.5.CustomizingThreadPoolExecutorAfterConstruction
MostoftheoptionspassedtotheThreadPoolExecutorconstructorscanalsobemodifiedafterconstructionviasetters (suchasthecorethreadpoolsize,maximumthreadpoolsize,keepͲalivetime,threadfactory,andrejectedexecution handler). If the Executor is created through one of the factory methods in Executors (except newSingleThreadExecutor),youcancasttheresulttoThread-PoolExecutortoaccessthesettersasinListing8.8.
Executors includes a factory method, unconfigurableExecutorService, which takes an existing ExecutorService andwrapsitwithoneexposingonlythemethodsofExecutorServicesoitcannotbefurtherconfigured.Unlikethe pooledimplementations,newSingleThreadExecutorreturnsanExecutorServicewrappedinthismanner,ratherthan a raw ThreadPoolExecutor. While a singleͲthreaded executor is actually implemented as a thread pool with one thread,italsopromisesnottoexecutetasksconcurrently.Ifsomemisguidedcodeweretoincreasethepoolsizeona singleͲthreadedexecutor,itwouldunderminetheintendedexecutionsemantics.
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 111
Listing8.7.CustomThreadBaseClass.
public class MyAppThread extends Thread {
public static final String DEFAULT_NAME = "MyAppThread"; private static volatile boolean debugLifecycle = false;
private static final AtomicInteger created = new AtomicInteger(); private static final AtomicInteger alive = new AtomicInteger();
private static final Logger log = Logger.getAnonymousLogger();
public MyAppThread(Runnable r) { this(r, DEFAULT_NAME); }
public MyAppThread(Runnable runnable, String name) {
super(runnable, name + "-" + created.incrementAndGet()); setUncaughtExceptionHandler(
new Thread.UncaughtExceptionHandler() {
public void uncaughtException(Thread t,
Throwable e) {
log.log(Level.SEVERE,
"UNCAUGHT in thread " + t.getName(), e);
}
});
}
public void run() {
// Copy debug flag to ensure consistent value throughout.
boolean debug = debugLifecycle;
if (debug) log.log(Level.FINE, "Created "+getName());
try {
alive.incrementAndGet();
super.run();
} finally {
alive.decrementAndGet();
if (debug) log.log(Level.FINE, "Exiting "+getName());
}
}
public static int getThreadsCreated() { return created.get(); }
public static int getThreadsAlive() { return alive.get(); }
public static boolean getDebug() { return debugLifecycle; }
public static void setDebug(boolean b) { debugLifecycle = b; }
}
Listing8.8.ModifyinganExecutorCreatedwiththeStandardFactories.
ExecutorService exec = Executors.newCachedThreadPool();
if (exec instanceof ThreadPoolExecutor)
((ThreadPoolExecutor) exec).setCorePoolSize(10);
else
throw new AssertionError("Oops, bad assumption");
Youcanusethistechniquewithyourownexecutorstopreventtheexecutionpolicyfrombeingmodified.Ifyouwillbe exposing an ExecutorService to code you don't trust not to modify it, you can wrap it with an unconfigurableExecutorService.
8.4.ExtendingThreadPoolExecutor
ThreadPoolExecutorwasdesignedforextension,providingseveral"hooks"forsubclassestooverridebeforeExecute, afterExecute,andterminatethatcanbeusedtoextendthebehaviorofThreadPoolExecutor.
The beforeExecute and afterExecute hooks are called in the thread that executes the task, and can be used for addinglogging,timing,monitoring,orstatisticsgathering.TheafterExecutehookiscalledwhetherthetaskcompletes byreturningnormallyfromrunorbythrowinganException.(IfthetaskcompleteswithanError,afterExecuteisnot called.)IfbeforeExecutethrowsaRuntimeException,thetaskisnotexecutedandafterExecuteisnotcalled.
Theterminatedhookiscalledwhenthethreadpoolcompletestheshutdownprocess,afteralltaskshavefinishedand allworkerthreadshaveshutdown.ItcanbeusedtoreleaseresourcesallocatedbytheExecutorduringitslifecycle, performnotificationorlogging,orfinalizestatisticsgathering.
8.4.1.Example:AddingStatisticstoaThreadPool
TimingThreadPool in Listing 8.9 shows a custom thread pool that uses before-Execute, afterExecute, and terminatedtoaddloggingandstatisticsgathering.Tomeasureatask'sruntime,beforeExecutemustrecordthestart timeandstoreitsomewhereafterExecutecanfindit.Becauseexecutionhooksarecalledinthethreadthatexecutes the task, a value placed in a ThreadLocal by beforeExecute can be retrieved by afterExecute. TimingThreadPool usesapairofAtomicLongstokeeptrackofthetotalnumberoftasksprocessedandthetotalprocessingtime,anduses theterminatedhooktoprintalogmessageshowingtheaveragetasktime.
112 JavaConcurrencyInPractice
Listing8.9.ThreadPoolExtendedwithLoggingandTiming.
public class TimingThreadPool extends ThreadPoolExecutor {
private final ThreadLocal<Long> startTime
= new ThreadLocal<Long>();
private final Logger log = Logger.getLogger("TimingThreadPool"); private final AtomicLong numTasks = new AtomicLong();
private final AtomicLong totalTime = new AtomicLong();
protected void beforeExecute(Thread t, Runnable r) {
super.beforeExecute(t, r);
log.fine(String.format("Thread %s: start %s", t, r));
startTime.set(System.nanoTime());
}
protected void afterExecute(Runnable r, Throwable t) {
try {
long endTime = System.nanoTime();
long taskTime = endTime - startTime.get();
numTasks.incrementAndGet();
totalTime.addAndGet(taskTime);
log.fine(String.format("Thread %s: end %s, time=%dns",
t, r, taskTime));
} finally {
super.afterExecute(r, t);
}
}
protected void terminated() {
try {
log.info(String.format("Terminated: avg time=%dns",
totalTime.get() / numTasks.get()));
} finally {
super.terminated();
}
}
}
8.5.ParallelizingRecursiveAlgorithms
ThepagerenderingexamplesinSection6.3wentthroughaseriesofrefinementsinsearchofexploitableparallelism.
The first attempt was entirely sequential; the second used two threads but still performed all the image downloads sequentially; the final version treated each image download as a separate task to achieve greater parallelism. Loops whose bodies contain nontrivial computation or perform potentially blocking I/O are frequently good candidates for parallelization,aslongastheiterationsareindependent.
If we have a loop whose iterations are independent and we don't need to wait for all of them to complete before proceeding, we can use an Executor to transform a sequential loop into a parallel one, as shown in processSequentiallyandprocessInParallelinListing8.10.
Listing8.10.TransformingSequentialExecutionintoParallelExecution.
void processSequentially(List<Element> elements) {
for (Element e : elements)
process(e);
}
void processInParallel(Executor exec, List<Element> elements) {
for (final Element e : elements)
exec.execute(new Runnable() {
public void run() { process(e); }
});
}
AcalltoprocessInParallelreturnsmorequicklythanacalltoprocessSequentiallybecauseitreturnsassoonasall thetasksarequeuedtotheExecutor,ratherthanwaitingforthemalltocomplete.Ifyouwanttosubmitasetoftasks andwaitforthemalltocomplete,youcanuse ExecutorService.invokeAll;toretrievetheresultsastheybecome available,youcanuseaCompletionService,asinRendereronpage130.
Sequentialloopiterationsaresuitableforparallelizationwheneachiterationisindependentoftheothersandthework doneineachiterationoftheloopbodyissignificantenoughtooffsetthecostofmanaginganewtask.
Loopparallelizationcanalsobeappliedtosomerecursivedesigns;thereareoftensequentialloopswithintherecursive algorithmthatcanbeparallelizedinthesamemannerasListing8.10.Theeasiercaseiswheneachiterationdoesnot require the results of the recursive iterations it invokes. For example, sequentialRecursive in Listing 8.11 does a
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 113
depthͲfirst traversal of a tree, performing a calculation on each node and placing the result in a collection. The transformedversion,parallelRecursive,alsodoesadepthͲfirsttraversal,butinsteadofcomputingtheresultaseach nodeisvisited,itsubmitsatasktocomputethenoderesult.
Listing8.11.TransformingSequentialTailǦrecursionintoParallelizedRecursion.
public<T> void sequentialRecursive(List<Node<T>> nodes, Collection<T> results) {
for (Node<T> n : nodes) {
results.add(n.compute());
sequentialRecursive(n.getChildren(), results);
}
}
public<T> void parallelRecursive(final Executor exec,
List<Node<T>> nodes,
final Collection<T> results) {
for (final Node<T> n : nodes) {
exec.execute(new Runnable() {
public void run() {
results.add(n.compute());
}
});
parallelRecursive(exec, n.getChildren(), results);
}
}
WhenparallelRecursivereturns,eachnodeinthetreehasbeenvisited(thetraversalisstillsequential:onlythecalls tocomputeareexecutedinparallel)andthecomputationforeachnodehasbeenqueuedtotheExecutor.Callersof parallelRecursive can wait for all the results by creating an Executor specific to the traversal and using shutdown andawaitTermination,asshowninListing8.12.
Listing8.12.WaitingforResultstobeCalculatedinParallel.
public<T> Collection<T> getParallelResults(List<Node<T>> nodes) throws InterruptedException {
ExecutorService exec = Executors.newCachedThreadPool();
Queue<T> resultQueue = new ConcurrentLinkedQueue<T>(); parallelRecursive(exec, nodes, resultQueue);
exec.shutdown();
exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
return resultQueue;
}
8.5.1.Example:APuzzleFramework
An appealing application of this technique is solving puzzles that involve finding a sequence of transformations from someinitialstatetoreachagoalstate,suchasthefamiliar"slidingblockpuzzles",[7]"HiͲQ","InstantInsanity",andother solitairepuzzles.
[7]Seehttp://www.puzzleworld.org/SlidingBlockPuzzles.
We define a "puzzle" as a combination of an initial position, a goal position, and a set of rules that determine valid moves.Therulesethastwoparts:computingthelistoflegalmovesfromagivenpositionandcomputingtheresultof applying a move to a position. Puzzle in Listing 8.13 shows our puzzle abstraction; the type parameters P and M
represent the classes for a position and a move. From this interface, we can write a simple sequential solver that searchesthepuzzlespaceuntilasolutionisfoundorthepuzzlespaceisexhausted.
Listing8.13.AbstractionforPuzzlesLikethe"SlidingBlocksPuzzle".
public interface Puzzle<P, M> {
P initialPosition();
boolean isGoal(P position);
Set<M> legalMoves(P position);
P move(P position, M move);
}
NodeinListing8.14representsapositionthathasbeenreachedthroughsomeseriesofmoves,holdingareferenceto themovethatcreatedthepositionandthepreviousNode.FollowingthelinksbackfromaNodeletsusreconstructthe sequenceofmovesthatledtothecurrentposition.
SequentialPuzzleSolverinListing8.15showsasequentialsolverforthepuzzleframeworkthatperformsadepthͲfirst searchofthepuzzlespace.Itterminateswhenitfindsasolution(whichisnotnecessarilytheshortestsolution).
Rewritingthesolvertoexploitconcurrencywouldallowustocomputenextmovesandevaluatethegoalconditionin parallel,sincetheprocessofevaluatingonemoveismostlyindependentofevaluatingothermoves.(Wesay"mostly"
114 JavaConcurrencyInPractice
because tasks share some mutable state, such as the set of seen positions.) If multiple processors are available, this couldreducethetimeittakestofindasolution.
ConcurrentPuzzleSolverinListing8.16usesaninnerSolverTaskclassthatextendsNodeandimplementsRunnable.
Most of the work is done in run: evaluating the set of possible next positions, pruning positions already searched, evaluating whether success has yet been achieved (by this task or by some other task), and submitting unsearched positionstoanExecutor.
To avoid infinite loops, the sequential version maintained a Set of previously searched positions; ConcurrentPuzzleSolverusesaConcurrentHashMapforthispurpose.Thisprovidesthreadsafetyandavoidstherace conditioninherentinconditionallyupdatingasharedcollectionbyusingputIfAbsenttoatomicallyaddapositiononly ifitwasnotpreviouslyknown.ConcurrentPuzzleSolverusestheinternalworkqueueofthethreadpoolinsteadof thecallstacktoholdthestateofthesearch.
Listing8.14.LinkNodeforthePuzzleSolverFramework.
@Immutable
static class Node<P, M> {
final P pos;
final M move;
final Node<P, M> prev;
Node(P pos, M move, Node<P, M> prev) {...}
List<M> asMoveList() {
List<M> solution = new LinkedList<M>();
for (Node<P, M> n = this; n.move != null; n = n.prev)
solution.add(0, n.move);
return solution;
}
}
The concurrent approach also trades one form of limitation for another that might be more suitable to the problem domain.ThesequentialversionperformsadepthͲfirstsearch,sothesearchisboundedbytheavailablestacksize.The concurrentversionperformsabreadthͲfirstsearchandisthereforefreeofthestacksizerestriction(butcanstillrunout ofmemoryifthesetofpositionstobesearchedoralreadysearchedexceedstheavailablememory).
In order to stop searching when we find a solution, we need a way to determine whether any thread has found a solutionyet.Ifwewanttoacceptthefirstsolutionfound,wealsoneedtoupdatethesolutiononlyifnoothertaskhas already found one. These requirements describe a sort of latch (see Section 5.5.1) and in particular, a resultͲbearing latch.WecouldeasilybuildablockingresultbearinglatchusingthetechniquesinChapter14,butitisofteneasierand lesserrorͲpronetouseexistinglibraryclassesratherthanlowͲlevellanguagemechanisms.ValueLatchinListing8.17
usesaCountDownLatchtoprovidetheneededlatchingbehavior,anduseslockingtoensurethatthesolutionissetonly once.
Eachtaskfirstconsultsthesolutionlatchandstopsifasolutionhasalreadybeenfound.Themainthreadneedstowait untilasolutionisfound; getValuein ValueLatch blocksuntilsomethread hassetthevalue.ValueLatchprovidesa way to hold a value such that only the first call actually sets the value, callers can test whether it has been set, and callerscanblockwaitingforittobeset.OnthefirstcalltosetValue,thesolutionisupdatedandtheCountDownLatchis decremented,releasingthemainsolverthreadfromgetValue.
ThefirstthreadtofindasolutionalsoshutsdowntheExecutor,topreventnewtasksfrombeingaccepted.Toavoid havingtodealwithRejectedExecutionException,therejectedexecutionhandlershouldbesettodiscardsubmitted tasks.Then,allunfinishedtaskseventuallyruntocompletionandanysubsequentattemptstoexecutenewtasksfail silently,allowingtheexecutortoterminate.(Ifthetaskstooklongertorun,wemightwanttointerrupttheminsteadof lettingthemfinish.)
5BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools 115
Listing8.15.SequentialPuzzleSolver.
public class SequentialPuzzleSolver<P, M> {
private final Puzzle<P, M> puzzle;
private final Set<P> seen = new HashSet<P>();
public SequentialPuzzleSolver(Puzzle<P, M> puzzle) {
this.puzzle = puzzle;
}
public List<M> solve() {
P pos = puzzle.initialPosition();
return search(new Node<P, M>(pos, null, null));
}
private List<M> search(Node<P, M> node) {
if (!seen.contains(node.pos)) {
seen.add(node.pos);
if (puzzle.isGoal(node.pos))
return node.asMoveList();
for (M move : puzzle.legalMoves(node.pos)) {
P pos = puzzle.move(node.pos, move);
Node<P, M> child = new Node<P, M>(pos, move, node);
List<M> result = search(child);
if (result != null)
return result;
}
}
return null;
}
static class Node<P, M> { /* Listing 8.14 */ }
}
Listing8.16.ConcurrentVersionofPuzzleSolver.
public class ConcurrentPuzzleSolver<P, M> {
private final Puzzle<P, M> puzzle;
private final ExecutorService exec;
private final ConcurrentMap<P, Boolean> seen;
final ValueLatch<Node<P, M>> solution
= new ValueLatch<Node<P, M>>();
...
public List<M> solve() throws InterruptedException {
try {
P p = puzzle.initialPosition();
exec.execute(newTask(p, null, null));
// block until solution found
Node<P, M> solnNode = solution.getValue();
return (solnNode == null) ? null : solnNode.asMoveList();
} finally {
exec.shutdown();
}
}
protected Runnable newTask(P p, M m, Node<P,M> n) {
return new SolverTask(p, m, n);
}
class SolverTask extends Node<P, M> implements Runnable {
...
public void run() {
if (solution.isSet()
|| seen.putIfAbsent(pos, true) != null)
return; // already solved or seen this position
if (puzzle.isGoal(pos))
solution.setValue(this);
else
for (M m : puzzle.legalMoves(pos))
exec.execute(
newTask(puzzle.move(pos, m), m, this));
}
}
}
116 JavaConcurrencyInPractice
Listing8.17.ResultǦbearingLatchUsedbyConcurrentPuzzleSolver.
@ThreadSafe
public class ValueLatch<T> {
@GuardedBy("this") private T value = null;
private final CountDownLatch done = new CountDownLatch(1);
public boolean isSet() {
return (done.getCount() == 0);
}
public synchronized void setValue(T newValue) {
if (!isSet()) {
value = newValue;
done.countDown();
}
}
public T getValue() throws InterruptedException {
done.await();
synchronized (this) {
return value;
}
}
}
ConcurrentPuzzleSolver does not deal well with the case where there is no solution: if all possible moves and positions have been evaluated and no solution has been found, solve waits forever in the call to getSolution. The sequentialversionterminatedwhenithadexhaustedthesearchspace,butgettingconcurrentprogramstoterminate cansometimesbemoredifficult.Onepossiblesolutionistokeepacountofactivesolvertasksandsetthesolutionto nullwhenthecountdropstozero,asinListing8.18.
Findingthesolutionmayalsotakelongerthanwearewillingtowait;thereareseveraladditionalterminationconditions wecouldimposeonthesolver.Oneisatimelimit;thisiseasilydonebyimplementingatimedgetValueinValueLatch (whichwouldusethetimedversionofawait),andshuttingdowntheExecutoranddeclaringfailureifgetValuetimes out.AnotherissomesortofpuzzleͲspecificmetricsuchassearchingonlyuptoacertainnumberofpositions.Orwecan provideacancellationmechanismandlettheclientmakeitsowndecisionaboutwhentostopsearching.
Listing8.18.SolverthatRecognizeswhenNoSolutionExists.
public class PuzzleSolver<P,M> extends ConcurrentPuzzleSolver<P,M> {
...
private final AtomicInteger taskCount = new AtomicInteger(0);
protected Runnable newTask(P p, M m, Node<P,M> n) {
return new CountingSolverTask(p, m, n);
}
class CountingSolverTask extends SolverTask {
CountingSolverTask(P pos, M move, Node<P, M> prev) {
super(pos, move, prev);
taskCount.incrementAndGet();
}
public void run() {
try {
super.run();
} finally {
if (taskCount.decrementAndGet() == 0)
solution.setValue(null);
}
}
}
}
Summary
TheExecutorframeworkisapowerfulandflexibleframeworkforconcurrentlyexecutingtasks.Itoffersanumberof tuning options, such as policies for creating and tearing down threads, handling queued tasks, and what to do with excesstasks,andprovidesseveralhooksforextendingitsbehavior.Asinmostpowerfulframeworks,however,there arecombinationsofsettingsthatdonotworkwelltogether;sometypesoftasksrequirespecificexecutionpolicies,and somecombinationsoftuningparametersmayproducestrangeresults.
5BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 117
Chapter9.GUIApplications
If you've tried to write even a simple GUI application using Swing, you know that GUI applications have their own peculiarthreadingissues.Tomaintainsafety,certaintasksmustrunintheSwingeventthread.Butyoucannotexecute longͲrunningtasksintheeventthread,lesttheUIbecomeunresponsive.AndSwingdatastructuresarenotthreadͲsafe, soyoumustbecarefultoconfinethemtotheeventthread.
Nearly all GUI toolkits, including Swing and SWT, are implemented as singleͲthreaded subsystems in which all GUI activityisconfinedtoasinglethread.IfyouarenotplanningtowriteatotallysingleͲthreadedprogram,therewillbe activitiesthatrunpartiallyinanapplicationthreadandpartiallyintheeventthread.Likemanyotherthreadingbugs, gettingthisdivisionwrongmaynotnecessarilymakeyourprogramcrashimmediately;instead,itcouldbehaveoddly underhardͲtoͲidentify conditions. EventhoughtheGUIframeworksthemselvesaresingleͲthreadedsubsystems,your applicationmaynotbe,andyoustillneedtoconsiderthreadingissuescarefullywhenwritingGUIcode.
9.1.WhyareGUIsSingleǦthreaded?
In the old days, GUI applications were singleͲthreaded and GUI events were processed from a "main event loop".
ModernGUIframeworksuseamodelthatisonlyslightlydifferent:theycreateadedicatedeventdispatchthread(EDT) forhandlingGUIevents.
SingleͲthreadedGUIframeworksarenotuniquetoJava;Qt,NextStep,MacOSCocoa,XWindows,andmanyothersare also singleͲthreaded. This is not for lack of trying; there have been many attempts to write multithreaded GUI frameworks,butbecauseofpersistentproblemswithraceconditionsanddeadlock,theyalleventuallyarrivedatthe singleͲthreaded event queue model in which a dedicated thread fetches events off a queue and dispatches them to applicationͲdefinedeventhandlers.(AWToriginallytriedtosupportagreaterdegreeofmultithreadedaccess,andthe decisiontomakeSwingsingleͲthreadedwasbasedlargelyonexperiencewithAWT.) Multithreaded GUI frameworks tend to be particularly susceptible to deadlock, partially because of the unfortunate interaction between input event processing and any sensible objectͲoriented modeling of GUI components. Actions initiatedbytheusertendto"bubbleup"fromtheOStotheapplicationͲamouseclickisdetectedbytheOS,isturned intoa"mouseclick"eventbythetoolkit,andiseventuallydeliveredtoanapplicationlistenerasahigherlevelevent suchasa"buttonpressed"event.Ontheotherhand,applicationͲinitiatedactions"bubbledown"fromtheapplication totheOSͲchangingthebackgroundcolorofacomponentoriginatesintheapplicationandisdispatchedtoaspecific componentclassandeventuallyintotheOSforrendering.CombiningthistendencyforactivitiestoaccessthesameGUI objectsintheoppositeorderwiththerequirementofmakingeachobjectthreadͲsafeyieldsarecipeforinconsistent lockordering,whichleadstodeadlock(seeChapter10).AndthisisexactlywhatnearlyeveryGUItoolkitdevelopment effortrediscoveredthroughexperience.
AnotherfactorleadingtodeadlockinmultithreadedGUIframeworksistheprevalenceofthemodelͲviewͲcontrol(MVC) pattern.Factoringuserinteractionsintocooperatingmodel,view,andcontrollerobjectsgreatlysimplifiesimplementing GUI applications, but again raises the risk of inconsistent lock ordering. The controller calls into the model, which notifiestheviewthatsomethinghaschanged.Butthecontrollercanalsocallintotheview,whichmayinturncallback into the model to query the model state. The result is again inconsistent lock ordering, with the attendant risk of deadlock.
Inhisweblog,[1]SunVPGrahamHamiltonnicelysumsupthechallenges,describingwhythemultithreadedGUItoolkitis oneoftherecurring"faileddreams"ofcomputerscience.
[1]http://weblogs.java.net/blog/kgh/archive/2004/10
I believe you can program successfully with multithreaded GUI toolkits if the toolkit is very carefully designed; if the toolkit exposes its locking methodology in gory detail; if you are very smart, very careful, and have a global understanding of the whole structure of the toolkit. If you get one of these things slightly wrong, things will mostly work,butyouwillgetoccasionalhangs(duetodeadlocks)orglitches(duetoraces).Thismultithreadedapproachworks bestforpeoplewhohavebeenintimatelyinvolvedinthedesignofthetoolkit.
Unfortunately,Idon'tthinkthissetofcharacteristicsscalestowidespreadcommercialuse.Whatyoutendtoendup withisnormalsmartprogrammersbuildingappsthatdon'tquiteworkreliablyforreasonsthatarenotatallobvious.So theauthorsgetverydisgruntledandfrustratedandusebadwordsonthepoorinnocenttoolkit.
118 JavaConcurrencyInPractice
SingleͲthreaded GUI frameworks achieve thread safety via thread confinement; all GUI objects, including visual componentsanddatamodels,areaccessedexclusivelyfromtheeventthread.Ofcourse,thisjustpushessomeofthe threadsafetyburdenbackontotheapplicationdeveloper,whomustmakesuretheseobjectsareproperlyconfined.
9.1.1.SequentialEventProcessing
GUI applications are oriented around processing fineͲgrained events such as mouse clicks, key presses, or timer expirations.Eventsareakindoftask;theeventhandlingmachineryprovidedbyAWTandSwingisstructurallysimilarto anExecutor.
BecausethereisonlyasinglethreadforprocessingGUItasks,theyareprocessedsequentiallyͲonetaskfinishesbefore thenextonebegins,andnotwotasksoverlap.KnowingthismakeswritingtaskcodeeasierͲyoudon'thavetoworry aboutinterferencefromothertasks.
Thedownsideofsequentialtaskprocessingisthatifonetasktakesalongtimetoexecute,othertasksmustwaituntilit isfinished.Ifthoseothertasksareresponsibleforrespondingtouserinputorprovidingvisualfeedback,theapplication willappeartohavefrozen.Ifalengthytaskisrunningintheeventthread,theusercannotevenclick"Cancel"because the cancel button listener is not called until the lengthy task completes. Therefore, tasks that execute in the event thread must return control to the event thread quickly. To initiate a longrunning task such as spellͲchecking a large document,searchingthefilesystem,orfetchingaresourceoveranetwork,youmustrunthattaskinanotherthreadso control can return quickly to the event thread. To update a progress indicator while a longͲrunning task executes or provide visual feedback when it completes, you again need to execute code in the event thread. This can get complicatedquickly.
9.1.2.ThreadConfinementinSwing
AllSwingcomponents(suchasJButtonandJTable)anddatamodelobjects(suchasTableModelandTReeModel)are confinedtotheeventthread,soanycodethataccessestheseobjectsmustrunintheeventthread.GUIobjectsarekept consistentnotbysynchronization,butbythreadconfinement.Theupsideisthattasksthatrunintheeventthreadneed not worry about synchronization when accessing presentation objects; the downside is that you cannot access presentationobjectsfromoutsidetheeventthreadatall.
TheSwingsingleͲthreadrule:Swingcomponentsandmodelsshouldbecreated,modified,andqueried onlyfromthe eventͲdispatchingthread.
Aswithallrules,thereareafewexceptions.AsmallnumberofSwingmethodsmaybecalledsafelyfromanythread; theseareclearlyidentifiedintheJavadocasbeingthreadͲsafe.OtherexceptionstothesingleͲthreadruleinclude: x SwingUtilities.isEventDispatchThread,whichdetermineswhetherthecurrentthreadistheeventthread; x SwingUtilities.invokeLater,whichschedulesaRunnableforexecutionontheeventthread(callablefrom anythread);
x SwingUtilities.invokeAndWait, which schedules a Runnable task for execution on the event thread and blocksthecurrentthreaduntilitcompletes(callableonlyfromanonͲGUIthread); x methodstoenqueuearepaintorrevalidationrequestontheeventqueue(callablefromanythread);and x methodsforaddingandremovinglisteners(canbecalledfromanythread,butlistenerswillalwaysbeinvoked intheeventthread).
The invokeLater and invokeAndWait methods function a lot like an Executor. In fact, it is trivial to implement the threadingͲrelatedmethodsfromSwingUtilitiesusingasingleͲthreadedExecutor,asshowninListing9.1.Thisisnot how SwingUtilities is actually implemented, as Swing predates the Executor framework, but is probably how it wouldbeifSwingwerebeingimplementedtoday.
TheSwingeventthreadcanbethoughtofasasingleͲthreadedExecutorthatprocessestasksfromtheeventqueue.As withthreadpools,sometimestheworkerthreaddiesandisreplacedbyanewone,butthisshouldbetransparentto tasks. Sequential, singleͲthreaded execution is a sensible execution policy when tasks are shortͲlived, scheduling predictabilityisnotimportant,oritisimperativethattasksnotexecuteconcurrently.
GuiExecutor in Listing 9.2 is an Executor that delegates tasks to SwingUtilities for execution. It could be implementedintermsofotherGUIframeworksaswell;forexample,SWTprovidestheDisplay.asyncExecmethod, whichissimilartoSwing'sinvokeLater.
5BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 119
9.2.ShortǦrunningGUITasks
In a GUI application, events originate in the event thread and bubble up to applicationͲprovided listeners, which will probablyperformsomecomputationthataffectsthepresentationobjects.Forsimple,shortͲrunningtasks,theentire action can stay in the event thread; for longerͲrunning tasks, some of the processing should be offloaded to another thread.
Inthesimplecase,confiningpresentationobjectstotheeventthreadiscompletelynatural.Listing9.3createsabutton whosecolorchangesrandomlywhenpressed.Whentheuserclicksonthebutton,thetoolkitdeliversanActionEvent intheeventthreadtoallregisteredactionlisteners.Inresponse,theactionlistenerpicksanewcolorandchangesthe button's background color. So the event originates in the GUI toolkit and is delivered to the application, and the applicationmodifiestheGUIinresponsetotheuser'saction.Controlneverhastoleavetheeventthread,asillustrated inFigure9.1.
Figure9.1.ControlFlowofaSimpleButtonClick.
This trivial example characterizes the majority of interactions between GUI applications and GUI toolkits. So long as tasksareshortͲlivedandaccessonlyGUIobjects(orotherthreadͲconfinedorthreadͲsafeapplicationobjects),youcan almosttotallyignorethreadingconcernsanddoeverythingfromtheeventthread,andtherightthinghappens.
120 JavaConcurrencyInPractice
Listing9.1.ImplementingSwingUtilitiesUsinganExecutor.
public class SwingUtilities {
private static final ExecutorService exec =
Executors.newSingleThreadExecutor(new SwingThreadFactory());
private static volatile Thread swingThread;
private static class SwingThreadFactory implements ThreadFactory {
public Thread newThread(Runnable r) {
swingThread = new Thread(r);
return swingThread;
}
}
public static boolean isEventDispatchThread() {
return Thread.currentThread() == swingThread;
}
public static void invokeLater(Runnable task) {
exec.execute(task);
}
public static void invokeAndWait(Runnable task)
throws InterruptedException, InvocationTargetException {
Future f = exec.submit(task);
try {
f.get();
} catch (ExecutionException e) {
throw new InvocationTargetException(e);
}
}
}
Listing9.2.ExecutorBuiltAtopSwingUtilities.
public class GuiExecutor extends AbstractExecutorService {
// Singletons have a private constructor and a public factory
private static final GuiExecutor instance = new GuiExecutor();
private GuiExecutor() { }
public static GuiExecutor instance() { return instance; }
public void execute(Runnable r) {
if (SwingUtilities.isEventDispatchThread())
r.run();
else
SwingUtilities.invokeLater(r);
}
// Plus trivial implementations of lifecycle methods
}
Listing9.3.SimpleEventListener.
final Random random = new Random();
final JButton button = new JButton("Change Color");
...
button.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
button.setBackground(new Color(random.nextInt()));
}
});
A slightly more complicated version of this same scenario, illustrated in Figure 9.2, involves the use of a formal data modelsuchasaTableModelortreeModel.Swingsplitsmostvisualcomponentsintotwoobjects,amodelandaview.
Thedatatobedisplayedresidesinthemodelandtherulesgoverninghowitisdisplayedresideintheview.Themodel objectscanfireeventsindicatingthatthemodeldatahaschanged,andviewssubscribetotheseevents.Whentheview receivesaneventindicatingthemodeldatamayhavechanged,itqueriesthemodelforthenewdataandupdatesthe display.Soinabuttonlistenerthatmodifiesthecontentsofatable,theactionlistenerwouldupdatethemodelandcall one of the fireXxx methods, which would in turn invoke the view's table model listeners, which would update the view. Again, control never leaves the event thread. (The Swing data model fireXxx methods always call the model listenersdirectlyratherthansubmittinganeweventtotheeventqueue,sothefireXxxmethodsmustbecalledonly fromtheeventthread.)
5BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 121
Figure9.2.ControlFlowwithSeparateModelandViewObjects.
9.3.LongǦrunningGUITasks
IfalltaskswereshortͲrunning(andtheapplicationhadnosignificantnonͲGUIportion),thentheentireapplicationcould runwithintheeventthreadandyouwouldn'thavetopayanyattentiontothreadsatall.However,sophisticatedGUI applicationsmayexecutetasksthatmaytakelongerthantheuseriswillingtowait,suchasspellchecking,background compilation,orfetchingremoteresources.ThesetasksmustruninanotherthreadsothattheGUIremainsresponsive whiletheyrun.
Swing makes it easy to have a task run in the event thread, but (prior to Java 6) doesn't provide any mechanism for helping GUI tasks execute code in other threads. But we don't need Swing to help us here: we can create our own ExecutorforprocessinglongͲrunningtasks.AcachedthreadpoolisagoodchoiceforlongͲrunningtasks;onlyrarelydo GUIapplicationsinitiatealargenumberoflongͲrunningtasks,sothereislittleriskofthepoolgrowingwithoutbound.
WestartwithasimpletaskthatdoesnotsupportcancellationorprogressindicationandthatdoesnotupdatetheGUI on completion, and then add those features one by one. Listing 9.4 shows an action listener, bound to a visual component,thatsubmitsalongͲrunningtasktoanExecutor.Despitethetwolayersofinnerclasses,havingaGUItask initiateataskinthismannerisfairlystraightforward:theUIactionlisteneriscalledintheeventthreadandsubmitsa Runnabletoexecuteinthethreadpool.
ThisexamplegetsthelongͲrunning taskoutofthe event threadina"fireandforget"manner,whichisprobablynot veryuseful.ThereisusuallysomesortofvisualfeedbackwhenalongͲrunningtaskcompletes.Butyoucannotaccess presentation objects from the background thread, so on completion the task must submit another task to run in the eventthreadtoupdatetheuserinterface.
Listing9.4.BindingaLongǦrunningTasktoaVisualComponent.
ExecutorService backgroundExec = Executors.newCachedThreadPool();
...
button.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
backgroundExec.execute(new Runnable() {
public void run() { doBigComputation(); }
});
}});
Listing9.5illustratestheobviouswaytodothis,whichisstartingtogetcomplicated;we'renowuptothreelayersof innerclasses.Theactionlistenerfirstdimsthebuttonandsetsalabelindicatingthatacomputationisinprogress,then submitsatasktothebackgroundexecutor.Whenthattaskfinishes,itqueuesanothertasktorunintheeventthread, whichreenablesthebuttonandrestoresthelabeltext.
122 JavaConcurrencyInPractice
Listing9.5.LongǦrunningTaskwithUserFeedback.
button.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
button.setEnabled(false);
label.setText("busy");
backgroundExec.execute(new Runnable() {
public void run() {
try {
doBigComputation();
} finally {
GuiExecutor.instance().execute(new Runnable() {
public void run() {
button.setEnabled(true);
label.setText("idle");
}
});
}
}
});
}
});
The task triggered when the button is pressed is composed of three sequential subtasks whose execution alternates betweentheeventthreadandthebackgroundthread.ThefirstsubtaskupdatestheuserinterfacetoshowthatalongͲ
running operation has begun and starts the second subtask in a background thread. Upon completion, the second subtaskqueuesthethirdsubtasktorunagainintheeventthread,whichupdatestheuserinterfacetoreflectthatthe operationhascompleted.Thissortof"threadhopping"istypicalofhandlinglongͲrunningtasksinGUIapplications.
9.3.1.Cancellation
Anytaskthattakeslongenoughtoruninanotherthreadprobablyalsotakeslongenoughthattheusermightwantto cancelit.Youcouldimplementcancellationdirectlyusingthreadinterruption,butitismucheasiertouseFuture,which wasdesignedtomanagecancellabletasks.
Whenyou callcancelon aFuturewithmayInterruptIfRunningsetto true,theFuture implementationinterrupts thethreadthatisexecutingthetaskifitiscurrentlyrunning.Ifyourtaskiswrittentoberesponsivetointerruption,it canreturnearlyifitiscancelled.Listing9.6illustratesataskthatpollsthethread'sinterruptedstatusandreturnsearly oninterruption.
Listing9.6.CancellingaLongǦrunningTask.
Future<?> runningTask = null; // thread-confined
...
startButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
if (runningTask != null) {
runningTask = backgroundExec.submit(new Runnable() {
public void run() {
while (moreWork()) {
if (Thread.currentThread().isInterrupted()) {
cleanUpPartialWork();
break;
}
doSomeWork();
}
}
});
};
}});
cancelButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
if (runningTask != null)
runningTask.cancel(true);
}});
BecauserunningTaskisconfinedtotheeventthread,nosynchronizationisrequiredwhensettingorcheckingit,and thestartbuttonlistenerensuresthatonlyonebackgroundtaskisrunningatatime.However,itwouldbebettertobe notifiedwhenthetaskcompletessothat,forexample,thecancelbuttoncouldbedisabled.Weaddressthisinthenext section.
9.3.2.ProgressandCompletionIndication
Using a Future to represent a longͲrunning task greatly simplified implementing cancellation. FutureTask also has a donehookthatsimilarlyfacilitatescompletionnotification.AfterthebackgroundCallablecompletes,doneiscalled.By
5BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 123
having done TRigger a completion task in the event thread, we can construct a BackgroundTask class providing an onCompletionhookthatiscalledintheeventthread,asshowninListing9.7.
BackgroundTask also supports progress indication. The compute method can call setProgress, indicating progress in numericalterms.ThiscausesonProgresstobe calledfromthe eventthread,whichcanupdate the userinterface to indicateprogressvisually.
ToimplementaBackgroundTaskyouneedonlyimplementcompute,whichiscalledinthebackgroundthread.Youalso havetheoptionofoverridingonCompletionandonProgress,whichareinvokedintheeventthread.
BasingBackgroundTaskonFutureTaskalsosimplifiescancellation.Ratherthanhavingtopollthethread'sinterrupted status, compute can call Future. is-Cancelled. Listing 9.8 recasts the example from Listing 9.6 using BackgroundTask.
9.3.3. SwingWorker
We'vebuiltasimpleframeworkusingFutureTaskandExecutortoexecutelongrunningtasksinbackgroundthreads without undermining the responsiveness of the GUI. These techniques can be applied to any singleͲthreaded GUI framework, not just Swing. In Swing, many of the features developed here are provided by the SwingWorker class, including cancellation, completion notification, and progress indication. Various versions of SwingWorker have been publishedinTheSwingConnectionandTheJavaTutorial,andanupdatedversionisincludedinJava6.
9.4.SharedDataModels
Swingpresentationobjects,includingdatamodelobjectssuchasTableModelortreeModel,areconfinedtotheevent thread.InsimpleGUIprograms,allthemutablestateisheldinthepresentationobjectsandtheonlythreadbesidesthe eventthreadisthemainthread.IntheseprogramsenforcingthesingleͲthreadruleiseasy:don'taccessthedatamodel orpresentationcomponentsfromthemainthread.Morecomplicatedprogramsmayuseotherthreadstomovedatato orfromapersistentstore,suchasafilesystemordatabase,soasnottocompromiseresponsiveness.
In the simplest case, the data in the data model is entered by the user or loaded statically from a file or other data sourceatapplicationstartup,inwhichcasethedataisnevertouchedbyanythreadotherthantheeventthread.But sometimesthepresentationmodelobjectisonlyaviewontoanotherdatasource,suchasadatabase,filesystem,or remoteservice.Inthiscase,morethanonethreadislikelytotouchthedataasitgoesintooroutoftheapplication.
124 JavaConcurrencyInPractice
Listing9.7.BackgroundTaskClassSupportingCancellation,CompletionNotification,andProgressNotification.
abstract class BackgroundTask<V> implements Runnable, Future<V> {
private final FutureTask<V> computation = new Computation(); private class Computation extends FutureTask<V> {
public Computation() {
super(new Callable<V>() {
public V call() throws Exception {
return BackgroundTask.this.compute() ;
}
});
}
protected final void done() {
GuiExecutor.instance().execute(new Runnable() {
public void run() {
V value = null;
Throwable thrown = null;
boolean cancelled = false;
try {
value = get();
} catch (ExecutionException e) {
thrown = e.getCause();
} catch (CancellationException e) {
cancelled = true;
} catch (InterruptedException consumed) {
} finally {
onCompletion(value, thrown, cancelled);
}
};
});
}
}
protected void setProgress(final int current, final int max) {
GuiExecutor.instance().execute(new Runnable() {
public void run() { onProgress(current, max); }
});
}
// Called in the background thread
protected abstract V compute() throws Exception;
// Called in the event thread
protected void onCompletion(V result, Throwable exception,
boolean cancelled) { }
protected void onProgress(int current, int max) { }
// Other Future methods forwarded to computation
}
Listing9.8.InitiatingaLongǦrunning,CancellableTaskwithBackgroundTask.
public void runInBackground(final Runnable task) {
startButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
class CancelListener implements ActionListener {
BackgroundTask<?> task;
public void actionPerformed(ActionEvent event) {
if (task != null)
task.cancel(true);
}
}
final CancelListener listener = new CancelListener();
listener.task = new BackgroundTask<Void>() {
public Void compute() {
while (moreWork() && !isCancelled())
doSomeWork();
return null;
}
public void onCompletion(boolean cancelled, String s,
Throwable exception) {
cancelButton.removeActionListener(listener);
label.setText("done");
}
};
cancelButton.addActionListener(listener);
backgroundExec.execute(task);
}
});
}
For example, you might display the contents of a remote file system using a tree control. You wouldn't want to enumeratetheentirefilesystembeforeyoucandisplaythetreecontrolthatwouldtaketoomuchtimeandmemory.
Instead, the tree can be lazily populated as nodes are expanded. Enumerating even a single directory on a remote volumecantakealongtime,soyoumaywanttodotheenumerationinabackgroundtask.Whenthebackgroundtask
5BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 125
completes, you have to get the data into the tree model somehow. This could be done by using a threadͲsafe tree model,by"pushing"thedatafromthebackgroundtasktotheeventthreadbypostingataskwithinvokeLater,orby havingtheeventthreadpolltoseeifthedataisavailable.
9.4.1.ThreadǦsafeDataModels
Aslongasresponsivenessisnotundulyaffectedbyblocking,theproblemofmultiplethreadsoperatingonthedatacan beaddressedwithathreadͲsafedatamodel.IfthedatamodelsupportsfineͲgrainedconcurrency,theeventthreadand background threads should be able to share it without responsiveness problems. For example, DelegatingVehicleTracker on page 65 uses an underlying ConcurrentHashMap whose retrieval operations offer a highdegreeofconcurrency.Thedownsideisthatitdoesnotofferaconsistentsnapshotofthedata,whichmayormay notbearequirement.ThreadͲsafedatamodelsmustalsogenerateeventswhenthemodelhasbeenupdated,sothat viewscanbeupdatedwhenthedatachanges.
Itmaysometimesbepossibletogetthreadsafety,consistencyandgoodresponsivenesswithaversioneddatamodel suchasCopyOnWriteArrayList[CPJ2.2.3.3].WhenyouacquireaniteratorforacopyͲonͲwritecollection,thatiterator traverses the collection as it existed when the iterator was created. However, copyͲonͲwrite collections offer good performance only when traversals greatly outnumber modifications, which would probably not be the case in, say, a vehicle tracking application. More specialized versioned data structures may avoid this restriction, but building versioned data structures that provide both efficient concurrent access and do not retain old versions of data longer thanneededisnoteasy,andthusshouldbeconsideredonlywhenotherapproachesarenotpractical.
9.4.2.SplitDataModels
From the perspective of the GUI, the Swing table model classes like TableModel and treeModel are the official repository for data to be displayed. However, these model objects are often themselves "views" of other objects managedbytheapplication.AprogramthathasbothapresentationͲdomainandanapplicationdomaindatamodelis saidtohaveasplitͲmodeldesign(Fowler,2005).
InasplitͲmodeldesign,thepresentationmodelisconfinedtotheeventthreadandtheothermodel,thesharedmodel, isthreadͲsafeandmaybeaccessedbyboththeeventthreadandapplicationthreads.Thepresentationmodelregisters listenerswiththesharedmodelsoitcanbenotifiedofupdates.Thepresentationmodelcanthenbeupdatedfromthe shared model by embedding a snapshot of the relevant state in the update message or by having the presentation modelretrievethedatadirectlyfromthesharedmodelwhenitreceivesanupdateevent.
Thesnapshotapproachissimple,buthaslimitations.Itworkswellwhenthedatamodelissmall,updatesarenottoo frequent,andthestructureofthetwomodelsissimilar.Ifthedatamodelislargeorupdatesareveryfrequent,orifone or both sides of the split contain information that is not visible to the other side, it can be more efficient to send incremental updates instead of entire snapshots. This approach has the effect of serializing updates on the shared model and recreating them in the event thread against the presentation model. Another advantage of incremental updatesisthatfinerͲgrainedinformationaboutwhatchanged canimprovetheperceivedqualityofthedisplayifonly onevehiclemoves,wedon'thavetorepainttheentiredisplay,justtheaffectedregions.
ConsiderasplitͲmodeldesignwhenadatamodelmustbesharedbymorethanonethreadandimplementingathreadͲ
safedatamodelwouldbeinadvisablebecauseofblocking,consistency,orcomplexityreasons.
9.5.OtherFormsofSingleǦthreadedSubsystems
Thread confinement is not restricted to GUIs: it can be used whenever a facility is implemented as a singleͲthreaded subsystem.Sometimesthreadconfinementisforcedonthedeveloperforreasonsthathavenothingtodowithavoiding synchronizationordeadlock.Forexample,somenativelibrariesrequirethatallaccesstothelibrary,evenloadingthe librarywithSystem.loadLibrary,bemadefromthesamethread.
Borrowing from the approach taken by GUI frameworks, you can easily create a dedicated thread or singleͲthreaded executorforaccessingthenativelibrary,andprovideaproxyobjectthatinterceptscallstothethreadͲconfinedobject andsubmitsthemastaskstothededicatedthread.FutureandnewSingleThreadExecutorworktogethertomakethis easy;theproxymethodcansubmitthetaskandimmediatelycallFuture.gettowaitfortheresult.(Iftheclasstobe threadͲconfinedimplementsaninterface,youcanautomatetheprocessofhavingeachmethodsubmitaCallabletoa backgroundthreadexecutorandwaitingfortheresultusingdynamicproxies.)
126 JavaConcurrencyInPractice
Summary
GUIframeworksarenearlyalwaysimplementedassingleͲthreadedsubsystemsinwhichallpresentationͲrelatedcode runs as tasks in an event thread. Because there is only a single event thread, longͲrunning tasks can compromise responsiveness and so should be executed in background threads. Helper classes like SwingWorker or the BackgroundTaskclassbuilthere,whichprovidesupportforcancellation,progressindication,andcompletionindication, cansimplifythedevelopmentoflongͲrunningtasksthathavebothGUIandnonͲGUIcomponents.
6BPartIII:Liveness,Performance,andTesting Ͳ 21BChapter9.GUIApplications 127
PartIII:Liveness,Performance,andTesting
Chapter10.AvoidingLivenessHazards
Chapter11.PerformanceandScalability
Chapter12.TestingConcurrentPrograms
128 JavaConcurrencyInPractice
Chapter10.AvoidingLivenessHazards
Thereisoftenatensionbetweensafetyandliveness.Weuselockingtoensurethreadsafety,butindiscriminateuseof locking can cause lockͲordering deadlocks. Similarly, we use thread pools and semaphores to bound resource consumption,butfailuretounderstandtheactivitiesbeingboundedcancauseresourcedeadlocks.Javaapplicationsdo notrecoverfromdeadlock,soitisworthwhiletoensurethatyourdesignprecludestheconditionsthatcouldcauseit.
Thischapterexploressomeofthecausesoflivenessfailuresandwhatcanbedonetopreventthem.
10.1.Deadlock
Deadlockisillustratedbytheclassic,ifsomewhatunsanitary,"diningphilosophers"problem.Fivephilosophersgoout forChinesefoodandareseatedatacirculartable.Therearefivechopsticks(notfivepairs),oneplacedbetweeneach pairofdiners.Thephilosophersalternatebetweenthinkingandeating.Eachneedstoacquiretwochopsticksforlong enough to eat, but can then put the chopsticks back and return to thinking. There are some chopstickͲmanagement algorithms that let everyone eat on a more or less timely basis (a hungry philosopher tries to grab both adjacent chopsticks,butifoneisnotavailable,putsdowntheonethatisavailableandwaitsaminuteorsobeforetryingagain), andsomethatcanresultinsomeorallofthephilosophersdyingofhunger(eachphilosopherimmediatelygrabsthe chopstick to his left and waits for the chopstick to his right to be available before putting down the left). The latter situation, where each has a resource needed by another and is waiting for a resource held by another, and will not releasetheonetheyholduntiltheyacquiretheonetheydon't,illustratesdeadlock.
When a thread holds a lock forever, other threads attempting to acquire that lock will block forever waiting. When threadAholdslockLandtriestoacquirelockM,butatthesametimethreadBholdsMandtriestoacquireL,both threads will wait forever. This situation is the simplest case of deadlock (or deadly embrace), where multiple threads waitforever duetoa cycliclocking dependency.(Thinkof the threadsasthenodesofa directed graphwhoseedges representtherelation"ThreadAiswaitingforaresourceheldbythreadB".Ifthisgraphiscyclical,thereisadeadlock.) Databasesystemsaredesignedtodetectandrecoverfromdeadlock.Atransactionmayacquiremanylocks,andlocks are held until the transaction commits. So it is quite possible, and in fact not uncommon, for two transactions to deadlock.Withoutintervention,theywouldwaitforever(holdinglocksthatareprobablyrequiredbyothertransactions aswell).Butthedatabaseserverisnotgoingtoletthishappen.Whenitdetectsthatasetoftransactionsisdeadlocked (whichitdoesbysearchingtheisͲwaitingͲforgraphforcycles),itpicksavictimandabortsthattransaction.Thisreleases the locks held by the victim, allowing the other transactions to proceed. The application can then retry the aborted transaction,whichmaybeabletocompletenowthatanycompetingtransactionshavecompleted.
TheJVMisnotnearlyashelpfulinresolvingdeadlocksasdatabaseserversare.WhenasetofJavathreadsdeadlock, that'stheendofthegamethosethreadsarepermanentlyoutofcommission.Dependingonwhatthosethreadsdo,the application may stall completely, or a particular subsystem may stall, or performance may suffer. The only way to restoretheapplicationtohealthistoabortandrestartitandhopethesamethingdoesn'thappenagain.
Like many other concurrency hazards, deadlocks rarely manifest themselves immediately. The fact that a class has a potentialdeadlockdoesn'tmeanthatiteverwilldeadlock,justthatitcan.Whendeadlocksdomanifestthemselves,itis oftenattheworstpossibletimeunderheavyproductionload.
10.1.1.LockǦorderingDeadlocks
LeftRightDeadlock in Listing 10.1 is at risk for deadlock. The leftRight and rightLeft methods each acquire the leftandrightlocks.IfonethreadcallsleftRightandanothercallsrightLeft,andtheiractionsareinterleavedas showninFigure10.1,theywilldeadlock.
Figure10.1.UnluckyTiminginLeftRightDeadlock.
6BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards 129
The deadlock in LeftRightDeadlock came about because the two threads attempted to acquire the same locks in a differentorder.Iftheyaskedforthelocksinthesameorder,therewouldbenocycliclockingdependencyandtherefore nodeadlock.IfyoucanguaranteethateverythreadthatneedslocksLandMatthesametimealwaysacquiresLandM
inthesameorder,therewillbenodeadlock.
AprogramwillbefreeoflockͲorderingdeadlocksifallthreadsacquirethelockstheyneedinafixedglobalorder.
Verifying consistent lock ordering requires a global analysis of your program's locking behavior. It is not sufficient to inspect code paths that acquire multiple locks individually; both leftRight and rightLeft are "reasonable" ways to acquirethetwolocks,theyarejustnotcompatible.Whenitcomestolocking,thelefthandneedstoknowwhatthe righthandisdoing.
Listing10.1.SimpleLockǦorderingDeadlock.
// Warning: deadlock-prone!
public class LeftRightDeadlock {
private final Object left = new Object();
private final Object right = new Object();
public void leftRight() {
synchronized (left) {
synchronized (right) {
doSomething();
}
}
}
public void rightLeft() {
synchronized (right) {
synchronized (left) {
doSomethingElse();
}
}
}
}
10.1.2.DynamicLockOrderDeadlocks
Sometimes it is not obvious that you have sufficient control over lock ordering to prevent deadlocks. Consider the harmlessͲlooking code in Listing 10.2 that transfers funds from one account to another. It acquires the locks on both Accountobjectsbeforeexecutingthetransfer,ensuringthatthebalancesareupdatedatomicallyandwithoutviolating invariantssuchas"anaccountcannothaveanegativebalance".
HowcanTRansferMoneydeadlock?Itmayappearasifallthethreadsacquiretheirlocksinthesameorder,butinfact the lock order depends on the order of arguments passed to transferMoney, and these in turn might depend on externalinputs.DeadlockcanoccuriftwothreadscalltransferMoneyatthesametime,onetransferringfromXtoY, andtheotherdoingtheopposite:
Listing10.2.DynamicLockǦorderingDeadlock.
// Warning: deadlock-prone!
public void transferMoney(Account fromAccount,
Account toAccount,
DollarAmount amount)
throws InsufficientFundsException {
synchronized (fromAccount) {
synchronized (toAccount) {
if (fromAccount.getBalance().compareTo(amount) < 0)
throw new InsufficientFundsException();
else {
fromAccount.debit(amount);
toAccount.credit(amount);
}
}
}
}
130 JavaConcurrencyInPractice
A: transferMoney(myAccount, yourAccount, 10);
B: transferMoney(yourAccount, myAccount, 20);
Withunluckytiming,AwillacquirethelockonmyAccountandwaitforthelockonyourAccount,whileBisholdingthe lockonyourAccountandwaitingforthelockonmyAccount.
DeadlockslikethisonecanbespottedthesamewayasinListing10.1lookfornestedlockacquisitions.Sincetheorder of arguments is out of our control, to fix the problem we must induce an ordering on the locks and acquire them accordingtotheinducedorderingconsistentlythroughouttheapplication.
OnewaytoinduceanorderingonobjectsistouseSystem.identityHashCode,whichreturnsthevaluethatwouldbe returnedbyObject.hashCode.Listing10.3showsaversionoftransferMoneythatusesSystem.identityHashCodeto inducealockordering.Itinvolvesafewextralinesofcode,buteliminatesthepossibilityofdeadlock.
In the rare case that two objects have the same hash code, we must use an arbitrary means of ordering the lock acquisitions,andthisreintroducesthepossibilityofdeadlock.Topreventinconsistentlockorderinginthiscase,athird
"tiebreaking"lockisused.ByacquiringthetieͲbreakinglockbeforeacquiringeitherAccountlock,weensurethatonly onethreadatatimeperformstheriskytaskofacquiringtwolocksinanarbitraryorder,eliminatingthepossibilityof deadlock(solongasthismechanismisusedconsistently).Ifhashcollisionswerecommon,thistechniquemightbecome a concurrency bottleneck (just as having a single, programͲwide lock would), but because hash collisions with System.identityHashCodearevanishinglyinfrequent,thistechniqueprovidesthatlastbitofsafetyatlittlecost.
Listing10.3.InducingaLockOrderingtoAvoidDeadlock.
private static final Object tieLock = new Object();
public void transferMoney(final Account fromAcct,
final Account toAcct,
final DollarAmount amount)
throws InsufficientFundsException {
class Helper {
public void transfer() throws InsufficientFundsException {
if (fromAcct.getBalance().compareTo(amount) < 0)
throw new InsufficientFundsException();
else {
fromAcct.debit(amount);
toAcct.credit(amount);
}
}
}
int fromHash = System.identityHashCode(fromAcct);
int toHash = System.identityHashCode(toAcct);
if (fromHash < toHash) {
synchronized (fromAcct) {
synchronized (toAcct) {
new Helper().transfer();
}
}
} else if (fromHash > toHash) {
synchronized (toAcct) {
synchronized (fromAcct) {
new Helper().transfer();
}
}
} else {
synchronized (tieLock) {
synchronized (fromAcct) {
synchronized (toAcct) {
new Helper().transfer();
}
}
}
}
}
IfAccounthasaunique,immutable,comparablekeysuchasanaccountnumber,inducingalockorderingiseveneasier: orderobjectsbytheirkey,thuseliminatingtheneedforthetieͲbreakinglock.
You may think we're overstating the risk of deadlock because locks are usually held only briefly, but deadlocks are a seriousprobleminrealsystems.AproductionapplicationmayperformbillionsoflockacquireͲreleasecyclesperday.
Onlyoneofthoseneedstobetimedjustwrongtobringtheapplicationtodeadlock,andevenathoroughloadͲtesting regimenmaynotdisclosealllatentdeadlocks.[1]DemonstrateDeadlockinListing10.4[2]deadlocksfairlyquicklyonmost systems.
6BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards 131
[1]Ironically,holdinglocksforshortperiodsoftime,asyouaresupposedtodotoreducelockcontention,increasesthelikelihoodthattestingwill notdiscloselatentdeadlockrisks.
[2]Forsimplicity,DemonstrateDeadlockignorestheissueofnegativeaccountbalances.
Listing10.4.DriverLoopthatInducesDeadlockUnderTypicalConditions.
public class DemonstrateDeadlock {
private static final int NUM_THREADS = 20;
private static final int NUM_ACCOUNTS = 5;
private static final int NUM_ITERATIONS = 1000000;
public static void main(String[] args) {
final Random rnd = new Random();
final Account[] accounts = new Account[NUM_ACCOUNTS];
for (int i = 0; i < accounts.length; i++)
accounts[i] = new Account();
class TransferThread extends Thread {
public void run() {
for (int i=0; i<NUM_ITERATIONS; i++) {
int fromAcct = rnd.nextInt(NUM_ACCOUNTS);
int toAcct = rnd.nextInt(NUM_ACCOUNTS);
DollarAmount amount =
new DollarAmount(rnd.nextInt(1000));
transferMoney(accounts[fromAcct],
accounts[toAcct], amount);
}
}
}
for (int i = 0; i < NUM_THREADS; i++)
new TransferThread().start();
}
}
10.1.3.DeadlocksBetweenCooperatingObjects
MultiplelockacquisitionisnotalwaysasobviousasinLeftRightDeadlockorTRansferMoney;thetwolocksneednot be acquired by the same method. Consider the cooperating classes in Listing 10.5, which might be used in a taxicab dispatchingapplication.Taxirepresentsanindividualtaxiwithalocationandadestination;Dispatcherrepresentsa fleetoftaxis.
While no method explicitly acquires two locks, callers of setLocation and getImage can acquire two locks just the same.IfathreadcallssetLocationinresponsetoanupdatefromaGPSreceiver,itfirstupdatesthetaxi'slocationand thencheckstoseeifithasreacheditsdestination.Ifithas,itinformsthedispatcherthatitneedsanewdestination.
Since both setLocation and notifyAvailable are synchronized, the thread calling setLocation acquires the Taxi lock and then the Dispatcher lock. Similarly, a thread calling getImage acquires the Dispatcher lock and then each Taxi lock (one at at time). Just as in LeftRightDeadlock, two locks are acquired by two threads in different orders, riskingdeadlock.
It was easy to spot the deadlock possibility in LeftRightDeadlock or transferMoney by looking for methods that acquiretwolocks.SpottingthedeadlockpossibilityinTaxiandDispatcherisalittleharder:thewarningsignisthatan alienmethod(definedonpage40)isbeingcalledwhileholdingalock.
Invoking an alien method with a lock held is asking for liveness trouble. The alien method might acquire other locks (riskingdeadlock)orblockforanunexpectedlylongtime,stallingotherthreadsthatneedthelockyouhold.
10.1.4.OpenCalls
Of course, Taxi and Dispatcher didn't know that they were each half of a deadlock waiting to happen. And they shouldn'thaveto;amethodcallisanabstractionbarrierintendedtoshieldyoufromthedetailsofwhathappensonthe otherside.Butbecauseyoudon'tknowwhatishappeningontheothersideofthecall,callinganalienmethodwitha lockheldisdifficulttoanalyzeandthereforerisky.
Callingamethodwithnolocksheldiscalledanopencall[CPJ2.4.1.3],andclassesthatrelyonopencallsaremorewellͲ
behavedandcomposablethanclassesthatmakecallswithlocksheld.Usingopencallstoavoiddeadlockisanalogousto using encapsulation to provide thread safety: while one can certainly construct a threadͲsafe program without any encapsulation,thethreadsafetyanalysisofaprogramthatmakeseffectiveuseofencapsulationisfareasierthanthat ofonethatdoesnot.Similarly,thelivenessanalysisofaprogramthatreliesexclusivelyonopencallsisfareasierthan thatofonethatdoesnot.Restrictingyourselftoopencallsmakesitfareasiertoidentifythecodepathsthatacquire multiplelocksandthereforetoensurethatlocksareacquiredinaconsistentorder.[3]
132 JavaConcurrencyInPractice
[3]Theneedtorelyonopen callsandcareful lockorderingreflectsthefundamentalmessinessofcomposingsynchronizedobjectsratherthan synchronizingcomposedobjects.
Listing10.5.LockǦorderingDeadlockBetweenCooperatingObjects.Don'tDothis.
// Warning: deadlock-prone!
class Taxi {
@GuardedBy("this") private Point location, destination; private final Dispatcher dispatcher;
public Taxi(Dispatcher dispatcher) {
this.dispatcher = dispatcher;
}
public synchronized Point getLocation() {
return location;
}
public synchronized void setLocation(Point location) {
this.location = location;
if (location.equals(destination))
dispatcher.notifyAvailable(this);
}
}
class Dispatcher {
@GuardedBy("this") private final Set<Taxi> taxis;
@GuardedBy("this") private final Set<Taxi> availableTaxis; public Dispatcher() {
taxis = new HashSet<Taxi>();
availableTaxis = new HashSet<Taxi>();
}
public synchronized void notifyAvailable(Taxi taxi) {
availableTaxis.add(taxi);
}
public synchronized Image getImage() {
Image image = new Image();
for (Taxi t : taxis)
image.drawMarker(t.getLocation());
return image;
}
}
TaxiandDispatcherinListing10.5canbeeasilyrefactoredtouseopencallsandthuseliminatethedeadlockrisk.This involvesshrinkingthesynchronizedblockstoguardonlyoperationsthatinvolvesharedstate,asinListing10.6.Very often, the cause of problems like those in Listing 10.5 is the use of synchronized methods instead of smaller synchronized blocks for reasons of compact syntax or simplicity rather than because the entire method must be guarded by a lock. (As a bonus, shrinking the synchronized block may also improve scalability as well; see Section 11.4.1foradviceonsizingsynchronizedblocks.)
Strive to use open calls throughout your program. Programs that rely on open calls are far easier to analyze for deadlockͲfreedomthanthosethatallowcallstoalienmethodswithlocksheld.
Restructuringasynchronizedblocktoallowopencallscansometimeshaveundesirableconsequences,sinceittakesan operationthatwasatomicandmakesitnotatomic.Inmanycases,thelossofatomicityisperfectlyacceptable;there's noreasonthatupdatingataxi'slocationandnotifyingthedispatcherthatitisreadyforanewdestinationneedbean atomicoperation.Inothercases,thelossofatomicityisnoticeablebutthesemanticchangesarestillacceptable.Inthe deadlockͲproneversion,getImageproducesacompletesnapshotofthefleetlocationsatthatinstant;intherefactored version,itfetchesthelocationofeachtaxiatslightlydifferenttimes.
Insomecases,however,thelossofatomicityisaproblem,andhereyouwillhavetouseanothertechniquetoachieve atomicity.Onesuch techniqueistostructureaconcurrentobjectsothatonlyonethreadcanexecutethe code path followingtheopencall.Forexample,whenshuttingdownaservice,youmaywanttowaitforinͲprogressoperationsto complete and then release resources used by the service. Holding the service lock while waiting for operations to complete is inherently deadlockͲprone, but releasing the service lock before the service is shut down may let other threads start new operations. The solution is to hold the lock long enough to update the service state to "shutting
6BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards 133
down"sothatotherthreadswantingtostartnewoperationsincludingshuttingdowntheserviceseethattheserviceis unavailable,anddonottry.Youcanthenwaitforshutdowntocomplete,knowingthatonlytheshutdownthreadhas accesstotheservicestateaftertheopencallcompletes.Thus,ratherthanusinglockingtokeeptheotherthreadsout ofacriticalsectionofcode,thistechniquereliesonconstructingprotocolssothatotherthreadsdon'ttrytogetin.
10.1.5.ResourceDeadlocks
Justasthreadscandeadlockwhentheyareeachwaitingforalockthattheotherholdsandwillnotrelease,theycan alsodeadlockwhenwaitingforresources.
Listing10.6.UsingOpenCallstoAvoidingDeadlockBetweenCooperatingObjects.
@ThreadSafe
class Taxi {
@GuardedBy("this") private Point location, destination; private final Dispatcher dispatcher;
...
public synchronized Point getLocation() {
return location;
}
public synchronized void setLocation(Point location) {
boolean reachedDestination;
synchronized (this) {
this.location = location;
reachedDestination = location.equals(destination);
}
if (reachedDestination)
dispatcher.notifyAvailable(this);
}
}
@ThreadSafe
class Dispatcher {
@GuardedBy("this") private final Set<Taxi> taxis;
@GuardedBy("this") private final Set<Taxi> availableTaxis;
...
public synchronized void notifyAvailable(Taxi taxi) {
availableTaxis.add(taxi);
}
public Image getImage() {
Set<Taxi> copy;
synchronized (this) {
copy = new HashSet<Taxi>(taxis);
}
Image image = new Image();
for (Taxi t : copy)
image.drawMarker(t.getLocation());
return image;
}
}
Sayyouhavetwopooledresources,suchasconnectionpoolsfortwodifferentdatabases.Resourcepoolsareusually implemented with semaphores (see Section 5.5.3) to facilitate blocking when the pool is empty. If a task requires connectionstobothdatabasesandthetworesourcesarenotalwaysrequestedinthesameorder,threadAcouldbe holding a connection to database D1 while waiting for a connection to database D2, and thread B could be holding a connectiontoD2whilewaitingforaconnectiontoD1.(Thelargerthepoolsare,thelesslikelythisistooccur;ifeach poolhasNconnections,deadlockrequiresNsetsofcyclicallywaitingthreadsandalotofunluckytiming.) AnotherformofresourceͲbaseddeadlockisthreadͲstarvationdeadlock.WesawanexampleofthishazardinSection 8.1.1,whereataskthatsubmitsataskandwaitsforitsresultexecutesinasingleͲthreadedExecutor.Inthatcase,the firsttaskwillwaitforever,permanentlystallingthattaskandallotherswaitingtoexecuteinthatExecutor.Tasksthat wait for the results of other tasks are the primary source of threadͲstarvation deadlock; bounded pools and interdependenttasksdonotmixwell.
10.2.AvoidingandDiagnosingDeadlocks
AprogramthatneveracquiresmorethanonelockatatimecannotexperiencelockͲorderingdeadlock.Ofcourse,thisis notalwayspractical,butifyoucangetawaywithit,it'salotlesswork.Ifyoumustacquiremultiplelocks,lockordering mustbeapartofyourdesign:trytominimizethenumberofpotentiallockinginteractions,andfollowanddocumenta lockͲorderingprotocolforlocksthatmaybeacquiredtogether.
134 JavaConcurrencyInPractice
InprogramsthatusefineͲgrainedlocking,audityourcodefordeadlockfreedomusingatwoͲpartstrategy:first,identify where multiple locks could be acquired (try to make this a small set), and then perform a global analysis of all such instances to ensure that lock ordering is consistent across your entire program. Using open calls wherever possible simplifiesthisanalysissubstantially.WithnononͲopencalls,findinginstanceswheremultiplelocksareacquiredisfairly easy,eitherbycoderevieworbyautomatedbytecodeorsourcecodeanalysis.
10.2.1.TimedLockAttempts
AnothertechniquefordetectingandrecoveringfromdeadlocksistousethetimedtryLockfeatureoftheexplicitLock classes(seeChapter13)insteadofintrinsiclocking.Whereintrinsiclockswaitforeveriftheycannotacquirethelock, explicitlocksletyouspecifyatimeoutafterwhichtryLockreturnsfailure.Byusingatimeoutthatismuchlongerthan you expect acquiring the lock to take, you can regain control when something unexpected happens. (Listing 13.3 on page280showsanalternativeimplementationoftransferMoneyusingthepolledtryLockwithretriesforprobabilistic deadlockavoidance.)
When a timed lock attempt fails, you do not necessarily know why. Maybe there was a deadlock; maybe a thread erroneouslyenteredaninfiniteloopwhileholdingthatlock;ormaybesomeactivityisjustrunningalotslowerthanyou expected.Still,atleastyouhavetheopportunitytorecordthatyourattemptfailed,loganyusefulinformationabout whatyouweretryingtodo,andrestartthecomputationsomewhatmoregracefullythankillingtheentireprocess.
Usingtimedlockacquisitiontoacquiremultiplelockscanbeeffectiveagainstdeadlockevenwhentimedlockingisnot usedconsistentlythroughouttheprogram.Ifalockacquisitiontimesout,youcanreleasethelocks,backoffandwait forawhile,andtryagain,possiblyclearingthedeadlockconditionandallowingtheprogramtorecover.(Thistechnique worksonlywhenthetwolocksareacquiredtogether;ifmultiplelocksareacquiredduetothenestingofmethodcalls, youcannotjustreleasetheouterlock,evenifyouknowyouholdit.)
10.2.2.DeadlockAnalysiswithThreadDumps
Whilepreventingdeadlocksismostlyyourproblem,theJVMcanhelpidentifythemwhentheydohappenusingthread dumps.Athreaddumpincludesastacktraceforeachrunningthread,similartothestacktracethataccompaniesan exception.Threaddumpsalsoincludelockinginformation,suchaswhichlocksareheldbyeachthread,inwhichstack frametheywereacquired,andwhichlockablockedthreadiswaitingtoacquire.[4]Beforegeneratingathreaddump, theJVMsearchestheisͲwaitingͲforgraphforcyclestofinddeadlocks.Ifitfindsone,itincludesdeadlockinformation identifyingwhichlocksandthreadsareinvolved,andwhereintheprogramtheoffendinglockacquisitionsare.
[4] This information is useful for debugging even when you don't have a deadlock; periodically triggering thread dumps lets you observe your program'slockingbehavior.
To trigger a thread dump, you can send the JVM process a SIGQUIT signal (kill -3) on Unix platforms, or pressthe Ctrl-\keyonUnixorCtrl-BreakonWindowsplatforms.ManyIDEscanrequestathreaddumpaswell.
If you are using the explicit Lock classes instead of intrinsic locking, Java 5.0 has no support for associating Lock information with the thread dump; explicit Locks do not show up at all in thread dumps. Java 6 does include thread dump support and deadlock detection with explicit Locks, but the information on where Locks are acquired is necessarily less precise than for intrinsic locks. Intrinsic locks are associated with the stack frame in which they were acquired;explicitLocksareassociatedonlywiththeacquiringthread.
Listing 10.7 shows portions of a thread dump taken from a production J2EE application. The failure that caused the deadlock involves three componentsa J2EE application, a J2EE container, and a JDBC driver, each from different vendors. (The names have been changed to protect the guilty.) All three were commercial products that had been throughextensivetestingcycles;eachhadabugthatwasharmlessuntiltheyallinteractedandcausedafatalserver failure.
We'veshownonlytheportionofthethreaddumprelevanttoidentifyingthedeadlock.TheJVMhasdonealotofwork for us in diagnosing the deadlockwhich locks are causing the problem, which threads are involved, which other locks they hold, and whether other threads are being indirectly inconvenienced. One thread holds the lock on the MumbleDBConnectionandiswaitingtoacquirethelockontheMumbleDBCallableStatement;theotherholdsthelock ontheMumbleDBCallableStatementandiswaitingforthelockontheMumbleDBConnection.
6BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards 135
Listing10.7.PortionofThreadDumpAfterDeadlock.
Found one Java-level deadlock:
=============================
"ApplicationServerThread":
waiting to lock monitor 0x080f0cdc (a MumbleDBConnection),
which is held by "ApplicationServerThread"
"ApplicationServerThread":
waiting to lock monitor 0x080f0ed4 (a MumbleDBCallableStatement), which is held by "ApplicationServerThread"
Java stack information for the threads listed above:
"ApplicationServerThread":
at MumbleDBConnection.remove_statement
- waiting to lock <0x650f7f30> (a MumbleDBConnection)
at MumbleDBStatement.close
- locked <0x6024ffb0> (a MumbleDBCallableStatement)
...
"ApplicationServerThread":
at MumbleDBCallableStatement.sendBatch
- waiting to lock <0x6024ffb0> (a MumbleDBCallableStatement) at MumbleDBConnection.commit
- locked <0x650f7f30> (a MumbleDBConnection)
...
TheJDBCdriverbeingusedhereclearlyhasalockͲorderingbug:differentcallchainsthroughtheJDBCdriveracquire multiple locks in different orders. But this problem would not have manifested itself were it not for another bug: multiplethreadsweretryingtousethesameJDBCConnectionatthesametime.Thiswasnothowtheapplicationwas supposedtoworkthedevelopersweresurprisedtoseethesameConnectionusedconcurrentlybytwothreads.There's nothingintheJDBCspecificationthatrequiresaConnectiontobethreadͲsafe,anditiscommontoconfineuseofa Connection to a single thread, as was intended here. This vendor tried to deliver a threadͲsafe JDBC driver, as evidencedbythesynchronizationonmultipleJDBCobjectswithinthedrivercode.Unfortunately,becausethevendor did not take lock ordering into account, the driver was prone to deadlock, but it was only the interaction of the deadlockͲprone driver and the incorrect Connection sharing by the application that disclosed the problem. Because neitherbugwasfatalinisolation,bothpersisteddespiteextensivetesting.
10.3.OtherLivenessHazards
While deadlock is the most widely encountered liveness hazard, there are several other liveness hazards you may encounter in concurrent programs including starvation, missed signals, and livelock. (Missed signals are covered in Section14.2.3.)
10.3.1.Starvation
Starvationoccurswhenathreadisperpetuallydeniedaccesstoresourcesitneedsinordertomakeprogress;themost commonlystarvedresourceisCPUcycles.StarvationinJavaapplicationscanbecausedbyinappropriateuseofthread priorities. It can also be caused by executing nonterminating constructs (infinite loops or resource waits that do not terminate)withalockheld,sinceotherthreadsthatneedthatlockwillneverbeabletoacquireit.
ThethreadprioritiesdefinedintheThreadAPIaremerelyschedulinghints.TheThreadAPIdefinestenprioritylevels thattheJVMcanmaptooperatingsystemschedulingprioritiesasitseesfit.ThismappingisplatformͲspecific,sotwo JavaprioritiescanmaptothesameOSpriorityononesystemanddifferentOSprioritiesonanother.Someoperating systemshavefewerthantenprioritylevels,inwhichcasemultipleJavaprioritiesmaptothesameOSpriority.
Operatingsystemschedulersgotogreatlengthstoprovideschedulingfairnessandlivenessbeyondthatrequiredbythe Java Language Specification. In most Java applications, all application threads have the same priority, Thread.
NORM_PRIORITY.Thethreadprioritymechanismisabluntinstrument,andit'snotalwaysobviouswhateffectchanging prioritieswillhave;boostingathread'sprioritymightdonothingormightalwayscauseonethreadtobescheduledin preferencetotheother,causingstarvation.
It is generally wise to resist the temptation to tweak thread priorities. As soon as you start modifying priorities, the behaviorofyourapplicationbecomesplatformͲspecificandyouintroducetheriskofstarvation.Youcanoftenspota program that is trying to recover from priority tweaking or other responsiveness problems by the presence of Thread.sleeporThread.yieldcallsinoddplaces,inanattempttogivemoretimetolowerͲprioritythreads.[5]
[5]ThesemanticsofThread.yield(andThread.sleep(0))areundefined[JLS17.9];theJVMisfreetoimplementthemasnoͲopsortreat themasschedulinghints.Inparticular,theyarenotrequiredtohavethesemanticsofsleep(0)onUnixsystemsͲputthecurrentthreadatthe endoftherunqueueforthatpriority,yieldingtootherthreadsofthesamepriorityͲthoughsomeJVMsimplementyieldinthisway.
136 JavaConcurrencyInPractice
Avoid the temptation to use thread priorities, since they increase platform dependence and can cause liveness problems.Mostconcurrentapplicationscanusethedefaultpriorityforallthreads.
10.3.2.PoorResponsiveness
One step removed from starvation is poor responsiveness, which is not uncommon in GUI applications using backgroundthreads.Chapter9developedaframeworkforoffloadinglongͲrunningtasksontobackgroundthreadssoas not to freeze the user interface. CPUͲintensive background tasks can still affect responsiveness because they can compete for CPU cycles with the event thread. This is one case where altering thread priorities makes sense; when computeͲintensivebackgroundcomputationswouldaffectresponsiveness.Iftheworkdonebyotherthreadsaretruly backgroundtasks,loweringtheirprioritycanmaketheforegroundtasksmoreresponsive.
Poor responsiveness can also be caused by poor lock management. If a thread holds a lock for a long time (perhaps whileiteratingalargecollectionandperformingsubstantialworkforeachelement),otherthreadsthatneedtoaccess thatcollectionmayhavetowaitaverylongtime.
10.3.3.Livelock
Livelockisaformoflivenessfailureinwhichathread,whilenotblocked,stillcannotmakeprogressbecauseitkeeps retrying an operation that will always fail. Livelock often occurs in transactional messaging applications, where the messaginginfrastructurerollsbackatransactionifamessagecannotbeprocessedsuccessfully,andputsitbackatthe head of the queue. If a bug in the message handler for a particular type of message causes it to fail, every time the messageisdequeuedandpassedtothebuggyhandler,thetransactionisrolledback.Sincethemessageisnowbackat theheadofthequeue,thehandleriscalledoverandoverwiththesameresult.(Thisissometimescalledthepoison message problem.) The message handling thread is not blocked, but it will never make progress either. This form of livelockoftencomesfromovereagererrorͲrecoverycodethatmistakenlytreatsanunrecoverableerrorasarecoverable one.
Livelockcanalsooccurwhenmultiplecooperatingthreadschangetheirstateinresponsetotheothersinsuchaway thatnothreadcanevermakeprogress.Thisissimilartowhathappenswhentwooverlypolitepeoplearewalkingin oppositedirectionsinahallway:eachstepsoutoftheother'sway,andnowtheyareagainineachother'sway.Sothey bothstepasideagain,andagain,andagain...
Thesolutionforthisvarietyoflivelockistointroducesomerandomnessintotheretrymechanism.Forexample,when twostationsinanEthernetnetworktrytosendapacketonthesharedcarrieratthesametime,thepacketscollide.The stationsdetectthecollision,andeachtriestosendtheirpacketagainlater.Iftheyeachretryexactlyonesecondlater, theycollideoverandover,andneitherpacketevergoesout,evenifthereisplentyofavailablebandwidth.Toavoid this,wemakeeachwait anamountoftime thatincludesarandomcomponent. (TheEthernetprotocolalsoincludes exponentialbackͲoffafterrepeatedcollisions,reducingbothcongestionandtheriskofrepeatedfailurewithmultiple collidingstations.)RetryingwithrandomwaitsandbackͲoffscanbeequallyeffectiveforavoidinglivelockinconcurrent applications.
Summary
Livenessfailuresareaseriousproblembecausethereisnowaytorecoverfromthemshortofabortingtheapplication.
The mostcommonform oflivenessfailureislockͲordering deadlock.Avoidinglockorderingdeadlockstartsatdesign time:ensurethatwhenthreadsacquiremultiplelocks,theydosoinaconsistentorder.Thebestwaytodothisisby usingopencallsthroughoutyourprogram.Thisgreatlyreducesthenumberofplaceswheremultiplelocksareheldat once,andmakesitmoreobviouswherethoseplacesare.
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 137
Chapter11.PerformanceandScalability
Oneoftheprimaryreasonstousethreadsistoimproveperformance.[1]Usingthreadscanimproveresourceutilization by letting applications more easily exploit available processing capacity, and can improve responsiveness by letting applicationsbeginprocessingnewtasksimmediatelywhileexistingtasksarestillrunning.
[1]Somemightarguethisistheonlyreasonweputupwiththecomplexitythreadsintroduce.
This chapter explores techniques for analyzing, monitoring, and improving the performance of concurrent programs.
Unfortunately, many of the techniques for improving performance also increase complexity, thus increasing the likelihood of safety and liveness failures. Worse, some techniques intended to improve performance are actually counterproductiveortradeonesortofperformanceproblemforanother.WhilebetterperformanceisoftendesirableͲ
andimprovingperformancecanbeverysatisfyingͲsafetyalwayscomesfirst.Firstmakeyourprogramright,thenmake itfastͲandthenonlyifyourperformancerequirementsandmeasurementstellyouitneedstobefaster.Indesigninga concurrentapplication,squeezingoutthelastbitofperformanceisoftentheleastofyourconcerns.
11.1.ThinkingaboutPerformance
Improvingperformancemeansdoingmoreworkwithfewerresources.Themeaningof"resources"canvary;foragiven activity,somespecificresourceisusuallyinshortestsupply,whetheritisCPUcycles,memory,networkbandwidth,I/O
bandwidth,databaserequests,diskspace,oranynumberofotherresources.Whentheperformanceofanactivityis limitedbyavailabilityofaparticularresource,wesayitisboundbythatresource:CPUͲbound,databaseͲbound,etc.
While the goal may be to improve performance overall, using multiple threads always introduces some performance costs compared to the singleͲthreaded approach. These include the overhead associated with coordinating between threads(locking,signaling,andmemorysynchronization),increasedcontextswitching,threadcreationandteardown, andschedulingoverhead.Whenthreadingisemployedeffectively,thesecostsaremorethanmadeupforbygreater throughput,responsiveness,orcapacity.Ontheotherhand,apoorlydesignedconcurrentapplicationcanperformeven worsethanacomparablesequentialone.[2]
[2]Acolleagueprovidedthisamusinganecdote:hehadbeeninvolvedinthetestingofanexpensiveandcomplexapplicationthatmanagedits workviaatunablethreadpool.Afterthesystemwascomplete,testingshowedthattheoptimalnumberofthreadsforthepoolwas...1.This shouldhavebeenobviousfromtheoutset;thetargetsystemwasasingleͲCPUsystemandtheapplicationwasalmostentirelyCPUͲbound.
Inusingconcurrencytoachievebetterperformance,wearetryingtodotwothings:utilizetheprocessingresourceswe have more effectively, and enable our program to exploit additional processing resources if they become available.
From a performance monitoring perspective, this means we are looking to keep the CPUs as busy as possible. (Of course,thisdoesn'tmeanburningcycleswithuselesscomputation;wewanttokeeptheCPUsbusywithusefulwork.)If theprogramiscomputeͲbound,thenwemaybeabletoincreaseitscapacitybyaddingmoreprocessors;ifitcan'teven keeptheprocessorswehavebusy,addingmorewon'thelp.ThreadingoffersameanstokeeptheCPU(s)"hotter"by decomposingtheapplicationsothereisalwaysworktobedonebyanavailableprocessor.
11.1.1.PerformanceVersusScalability
Applicationperformancecanbemeasuredinanumberofways,suchasservicetime,latency,throughput,efficiency, scalability, or capacity. Some of these (service time, latency) aremeasures of "how fast" a given unit of work can be processedoracknowledged;others(capacity,throughput)aremeasuresof"howmuch"workcanbeperformedwitha givenquantityofcomputingresources.
Scalability describes the ability to improve throughput or capacity when additional computing resources (such as additionalCPUs,memory,storage,orI/Obandwidth)areadded.
Designing and tuning concurrent applications for scalability can be very different from traditional performance optimization.Whentuningforperformance,thegoalisusuallytodothesameworkwithlesseffort,suchasbyreusing previouslycomputedresultsthroughcachingorreplacinganO(n2)algorithmwithanO(nlogn)one.Whentuningfor scalability, you are instead trying to find ways to parallelize the problem so you can take advantage of additional processingresourcestodomoreworkwithmoreresources.
ThesetwoaspectsofperformanceͲhowfastandhowmuchͲarecompletelyseparate,andsometimesevenatodds with each other. In order to achieve higher scalability or better hardware utilization, we often end up increasing the amountofworkdonetoprocesseachindividualtask,suchaswhenwedividetasksintomultiple"pipelined"subtasks.
138 JavaConcurrencyInPractice
Ironically,manyofthetricksthatimproveperformanceinsingleͲthreadedprogramsarebadforscalability(seeSection 11.4.4foranexample).
ThefamiliarthreeͲtierapplicationmodelͲinwhichpresentation,businesslogic,andpersistenceareseparatedandmay be handled by different systems Ͳ illustrates how improvements in scalability often come at the expense of performance. A monolithic application where presentation, business logic, and persistence are intertwined would almost certainly provide better performance for the first unit of work than would a wellͲfactored multitier implementation distributed over multiple systems. How could it not? The monolithic application would not have the networklatencyinherentinhandingofftasksbetweentiers,norwouldithavetopaythecostsinherentinseparatinga computational process into distinct abstracted layers (such as queuing overhead, coordination overhead, and data copying).
However, when the monolithic system reaches its processing capacity, we could have a serious problem: it may be prohibitivelydifficulttosignificantlyincreasecapacity.Soweoftenaccepttheperformancecostsoflongerservicetime orgreatercomputingresourcesusedperunitofworksothatourapplicationcanscaletohandlegreaterloadbyadding moreresources.
Ofthevariousaspectsofperformance,the"howmuch"aspectsͲscalability,throughput,andcapacityͲareusuallyof greater concern for server applications than the "how fast" aspects. (For interactive applications, latency tends to be more important, so that users need not wait for indications of progress and wonder what is going on.) This chapter focusesprimarilyonscalabilityratherthanrawsingleͲthreadedperformance.
11.1.2.EvaluatingPerformanceTradeoffs
Nearly all engineering decisions involve some form of tradeoff. Using thicker steel in a bridge span may increase its capacityandsafety,butalsoitsconstructioncost.Whilesoftwareengineeringdecisionsdon'tusuallyinvolvetradeoffs between money and risk to human life, we often have less information with which to make the right tradeoffs. For example, the "quicksort" algorithm is highly efficient for large data sets, but the less sophisticated "bubble sort" is actuallymoreefficientforsmalldatasets.Ifyouareaskedtoimplementanefficientsortroutine,youneed toknow somethingaboutthesizesofdatasetsitwillhavetoprocess,alongwithmetricsthattellyouwhetheryouaretryingto optimizeaverageͲcasetime,worstͲcasetime,orpredictability.Unfortunately,thatinformationisoftennotpartofthe requirements given to the author of a library sort routine. This is one of the reasons why most optimizations are premature:theyareoftenundertakenbeforeaclearsetofrequirementsisavailable.
Avoidprematureoptimization.Firstmakeitright,thenmakeitfastͲifitisnotalreadyfastenough.
When making engineering decisions, sometimes you are trading one form of cost for another (service time versus memoryconsumption);sometimesyouaretradingcostforsafety.Safetydoesn'tnecessarilymeanrisktohumanlives, asitdidinthebridgeexample.ManyperformanceoptimizationscomeatthecostofreadabilityormaintainabilityͲthe more "clever" or nonͲobvious code is, the harder it is to understand and maintain. Sometimes optimizations entail compromisinggoodobjectͲorienteddesignprinciples,suchasbreakingencapsulation;sometimestheyinvolvegreater riskoferror,becausefasteralgorithmsareusuallymorecomplicated.(Ifyoucan'tspotthecostsorrisks,youprobably haven'tthoughtitthroughcarefullyenoughtoproceed.)
Mostperformancedecisionsinvolvemultiplevariablesandarehighlysituational.Beforedecidingthatoneapproachis
"faster"thananother,askyourselfsomequestions:
x Whatdoyoumeanby"faster"?
x Underwhatconditionswillthisapproachactuallybefaster?Underlightorheavyload?Withlargeorsmalldata sets?Canyousupportyouranswerwithmeasurements?
x How often are these conditions likely to arise in your situation? Can you support your answer with measurements?
x Isthiscodelikelytobeusedinothersituationswheretheconditionsmaybedifferent?
x What hidden costs, such as increased development or maintenance risk, are you trading for this improved performance?Isthisagoodtradeoff?
TheseconsiderationsapplytoanyperformanceͲrelatedengineeringdecision,butthisisabookaboutconcurrency.Why arewerecommendingsuchaconservativeapproachtooptimization?Thequestforperformanceisprobablythesingle greatest source of concurrency bugs. The belief that synchronization was "too slow" led to many cleverͲlooking but dangerous idioms for reducing synchronization (such as doubleͲchecked locking, discussed in Section 16.2.4), and is oftencitedasanexcusefornotfollowingtherulesregardingsynchronization.Becauseconcurrencybugsareamongthe
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 139
most difficult to track down and eliminate, however, anything that risks introducing them must be undertaken very carefully.
Worse, when you trade safety for performance, you may get neither. Especially when it comes to concurrency, the intuitionofmanydevelopersaboutwhereaperformanceproblemliesorwhichapproachwillbefasterormorescalable is often incorrect. It is therefore imperative that any performance tuning exercise be accompanied by concrete performance requirements (so you know both when to tune and when to stop tuning) and with a measurement program in place using a realistic configuration and load profile. Measure again after tuning to verify that you've achieved the desired improvements. The safety and maintenance risks associated with many optimizations are bad enoughͲyoudon'twanttopaythesecostsifyoudon'tneedtoͲandyoudefinitelydon'twanttopaythemifyoudon't evengetthedesiredbenefit.
Measure,don'tguess.
There are sophisticated profiling tools on the market for measuring performance and tracking down performance bottlenecks,butyoudon'thavetospendalotofmoneytofigureoutwhatyourprogramisdoing.Forexample,thefree perfbarapplicationcangiveyouagoodpictureofhowbusytheCPUsare,andsinceyourgoalisusuallytokeepthe CPUsbusy,thisisaverygoodwaytoevaluatewhetheryouneedperformancetuningorhoweffectiveyourtuninghas been.
11.2.Amdahl'sLaw
SomeproblemscanbesolvedfasterwithmoreresourcesͲthemoreworkersavailableforharvestingcrops,thefaster the harvest can be completed. Other tasks are fundamentally serial Ͳ no number of additional workers will make the cropsgrowanyfaster.Ifoneofourprimaryreasonsforusingthreadsistoharnessthepowerofmultipleprocessors,we mustalsoensurethattheproblemisamenabletoparalleldecompositionandthatourprogrameffectivelyexploitsthis potentialforparallelization.
Mostconcurrentprogramshavealotincommonwithfarming,consistingofamixofparallelizableandserialportions.
Amdahl'slawdescribeshowmuchaprogramcantheoreticallybespedupbyadditionalcomputingresources,basedon the proportion of parallelizable and serial components. If F is the fraction of the calculation that must be executed serially,thenAmdahl'slawsaysthatonamachinewithNprocessors,wecanachieveaspeedupofatmost: ͳ
ܵ݁݁݀ݑ
ܨ ͳ െ ܨ
ܰ
AsNapproachesinfinity,themaximumspeedupconvergesto1/F,meaningthataprograminwhichfiftypercentofthe processing must be executed serially can be sped up only by a factor of two, regardless of how many processors are available, and a program in which ten percent must be executed serially can be sped up by at most a factor of ten.
Amdahl'slawalsoquantifiestheefficiency costofserialization. With tenprocessors,aprogramwith 10%serialization canachieveatmostaspeedupof5.3(at53%utilization),andwith100processorsitcanachieveatmostaspeedupof 9.2(at9%utilization).IttakesalotofinefficientlyutilizedCPUstonevergettothatfactoroften.
Figure11.1showsthemaximumpossibleprocessorutilizationforvaryingdegreesofserialexecutionandnumbersof processors. (Utilization is defined as the speedup divided by the number of processors.) It is clear that as processor counts increase, even a small percentage of serialized execution limits how much throughput can be increased with additionalcomputingresources.
140 JavaConcurrencyInPractice
Figure11.1.MaximumUtilizationUnderAmdahl'sLawforVariousSerializationPercentages.
Chapter6exploredidentifyinglogicalboundariesfordecomposingapplicationsintotasks.Butinordertopredictwhat kind of speedup is possible from running your application on a multiprocessor system, you also need to identify the sourcesofserializationinyourtasks.
ImagineanapplicationwhereNthreadsexecutedoWorkinListing11.1,fetchingtasksfromasharedworkqueueand processingthem;assumethattasksdonotdependontheresultsorsideeffectsofothertasks.Ignoringforamoment how the tasks get onto the queue, how well will this application scale as we add processors? At first glance, it may appear that the application is completely parallelizable: tasks do not wait for each other, and the more processors available,themoretaskscanbeprocessedconcurrently.However,thereisaserialcomponentaswellͲfetchingthetask from the work queue. The work queue is shared by all the worker threads, and it will require some amount of synchronization to maintain its integrity in the face of concurrent access. If locking is used to guard the state of the queue,thenwhileonethreadisdequeingatask,otherthreadsthatneedtodequeuetheirnexttaskmustwaitͲandthis iswheretaskprocessingisserialized.
The processing time of a single task includes not only the time to execute the task Runnable, but also the time to dequeuethetaskfromthesharedworkqueue.IftheworkqueueisaLinkedBlockingQueue,thedequeueoperation mayblocklessthanwith asynchronizedLinkedListbecauseLinkedBlockingQueueusesamorescalablealgorithm, butaccessinganyshareddatastructurefundamentallyintroducesanelementofserializationintoaprogram.
This example also ignores another common source of serialization: result handling. All useful computations produce some sortof result or side effectif not, they can be eliminated as dead code. Since Runnable provides for no explicit resulthandling,thesetasksmusthavesomesortofsideeffect,saywritingtheirresultstoalogfileorputtingthemina data structure. Log files and result containers are usually shared by multiple worker threads and therefore are also a sourceofserialization.Ifinsteadeachthreadmaintainsitsowndatastructureforresultsthataremergedafterallthe tasksareperformed,thenthefinalmergeisasourceofserialization.
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 141
Listing11.1.SerializedAccesstoaTaskQueue.
public class WorkerThread extends Thread {
private final BlockingQueue<Runnable> queue;
public WorkerThread(BlockingQueue<Runnable> queue) {
this.queue = queue;
}
public void run() {
while (true) {
try {
Runnable task = queue.take();
task.run();
} catch (InterruptedException e) {
break; /* Allow thread to exit */
}
}
}
}
Allconcurrentapplicationshavesomesourcesofserialization;ifyouthinkyoursdoesnot,lookagain.
11.2.1.Example:SerializationHiddeninFrameworks
Toseehowserializationcanbehiddeninthestructureofanapplication,wecancomparethroughputasthreadsare added and infer differences in serialization based on observed differences in scalability. Figure 11.2 shows a simple application in which multiple threads repeatedly remove an element from a shared Queue and process it, similar to Listing11.1.TheprocessingstepinvolvesonlythreadͲlocalcomputation.Ifathreadfindsthequeueisempty,itputsa batchofnewelementsonthequeuesothatotherthreadshavesomethingtoprocessontheirnextiteration.Accessing thesharedqueueclearlyentailssomedegreeofserialization,buttheprocessingstepisentirelyparallelizablesinceit involvesnoshareddata.
Figure11.2.ComparingQueueImplementations.
[Viewfullsizeimage]
ThecurvesinFigure11.2comparethroughputfortwothreadͲsafeQueueimplementations:aLinkedListwrappedwith synchronizedList,andaConcurrentLinkedQueue.Thetestswererunonan8ͲwaySparcV880systemrunningSolaris.
Whileeachrunrepresentsthesameamountof"work",wecanseethatmerelychangingqueueimplementationscan haveabigimpactonscalability.
The throughput of ConcurrentLinkedQueue continues to improve until it hits the number of processors and then remains mostly constant. On the other hand, the throughput of the synchronized LinkedList shows some improvementuptothreethreads,butthenfallsoffassynchronizationoverheadincreases.Bythetimeitgetstofouror fivethreads,contentionissoheavythateveryaccesstothequeuelockiscontendedandthroughputisdominatedby contextswitching.
The difference in throughput comes from differing degrees of serialization between the two queue implementations.
ThesynchronizedLinkedListguardstheentirequeuestatewithasinglelockthatisheldforthedurationoftheoffer or remove call; ConcurrentLinkedQueue uses a sophisticated nonͲblocking queue algorithm (see Section 15.4.2) that
142 JavaConcurrencyInPractice
uses atomic references to update individual link pointers. In one, the entire insertion or removal is serialized; in the other,onlyupdatestoindividualpointersareserialized.
11.2.2.ApplyingAmdahl'sLawQualitatively
Amdahl's law quantifies the possible speedup when more computing resources are available, if we can accurately estimatethefractionofexecutionthatisserialized.Althoughmeasuringserializationdirectlycanbedifficult,Amdahl's lawcanstillbeusefulwithoutsuchmeasurement.
Since our mental models are influenced by our environment, many of us are used to thinking that a multiprocessor systemhastwoorfourprocessors,ormaybe(ifwe'vegotabigbudget)asmanyasafewdozen,becausethisisthe technologythathasbeenwidelyavailableinrecentyears.ButasmulticoreCPUsbecomemainstream,systemswillhave hundreds or even thousands of processors. [3] Algorithms that seem scalable on a fourͲway system may have hidden scalabilitybottlenecksthathavejustnotyetbeenencountered.
[3]Marketupdate:atthiswriting,SunisshippinglowͲendserversystemsbasedonthe8ͲcoreNiagaraprocessor,andAzulisshippinghighͲend serversystems(96,192,and384Ͳway)basedonthe24ͲcoreVegaprocessor.
When evaluating an algorithm, thinking "in the limit" about what would happen with hundreds or thousands of processors can offer some insight into where scaling limits might appear. For example, Sections 11.4.2 and 11.4.3
discusstwotechniquesforreducinglockgranularity:locksplitting(splittingonelockintotwo)andlockstriping(splitting onelockintomany).LookingatthemthroughthelensofAmdahl'slaw,weseethatsplittingalockintwodoesnotget usveryfartowardsexploitingmanyprocessors,butlockstripingseemsmuchmorepromisingbecausethesizeofthe stripe set can be increased as processor count increases. (Of course, performance optimizations should always be consideredinlightofactualperformancerequirements;insomecases,splittingalockintwomaybeenoughtomeet therequirements.)
11.3.CostsIntroducedbyThreads
SingleͲthreadedprograms incurneitherschedulingnorsynchronizationoverhead,andneednotuselockstopreserve the consistency of data structures. Scheduling and interthread coordination have performance costs; for threads to offeraperformanceimprovement,theperformancebenefitsofparallelizationmustoutweighthecostsintroducedby concurrency.
11.3.1.ContextSwitching
Ifthemainthreadistheonlyschedulablethread,itwillalmostneverbescheduledout.Ontheotherhand,ifthereare more runnable threads than CPUs, eventually the OS will preempt one thread so that another can use the CPU. This causesacontextswitch,whichrequiressavingtheexecutioncontextofthecurrentlyrunningthreadandrestoringthe executioncontextofthenewlyscheduledthread.
Contextswitchesarenotfree;threadschedulingrequiresmanipulatingshareddatastructuresintheOSandJVM.The OSandJVMusethesameCPUsyourprogramdoes;moreCPUtimespentinJVMandOScodemeanslessisavailablefor yourprogram.ButOSandJVMactivityisnottheonlycostofcontextswitches.Whenanewthreadisswitchedin,the dataitneedsisunlikelytobeinthelocalprocessorcache,soacontextswitchcausesaflurryofcachemisses,andthus threads run a little more slowly when they are first scheduled. This is one of the reasons that schedulers give each runnablethreadacertainminimumtimequantumevenwhenmanyotherthreadsarewaiting:itamortizesthecostof the context switch and its consequences over more uninterrupted execution time, improving overall throughput (at somecosttoresponsiveness).
Listing11.2.SynchronizationthathasNoEffect.
synchronized (new Object()) {
// do something
}
Whenathreadblocksbecauseitiswaitingforacontendedlock,theJVMusuallysuspendsthethreadandallowsittobe switchedout.Ifthreadsblockfrequently,theywillbeunabletousetheirfullschedulingquantum.Aprogramthatdoes more blocking (blocking I/O, waiting for contended locks, or waiting on condition variables) incurs more context switches than one that is CPUͲbound, increasing scheduling overhead and reducing throughput. (NonͲblocking algorithmscanalsohelpreducecontextswitches;seeChapter15.)
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 143
Theactualcostofcontextswitchingvariesacrossplatforms,butagoodruleofthumbisthatacontextswitchcoststhe equivalentof5,000to10,000clockcycles,orseveralmicrosecondsonmostcurrentprocessors.
The vmstat command on Unix systems and the perfmon tool on Windows systems report the number of context switchesandthepercentageoftimespentinthekernel.Highkernelusage(over10%)oftenindicatesheavyscheduling activity,whichmaybecausedbyblockingduetoI/Oorlockcontention.
11.3.2.MemorySynchronization
The performance cost of synchronization comes from several sources. The visibility guarantees provided by synchronized and volatile may entail using special instructions called memory barriers that can flush or invalidate caches, flush hardware write buffers, and stall execution pipelines. Memory barriers may also have indirect performance consequences because they inhibit other compiler optimizations; most operations cannot be reordered withmemorybarriers.
When assessing the performance impact of synchronization, it is important to distinguish between contended and uncontended synchronization. The synchronized mechanism is optimized for the uncontended case (volatile is always uncontended), and at this writing, the performance cost of a "fastͲpath" uncontended synchronization ranges from 20 to 250 clock cycles for most systems. While this is certainly not zero, the effect of needed, uncontended synchronizationisrarelysignificantinoverallapplicationperformance,andthealternativeinvolvescompromisingsafety andpotentiallysigningyourself(oryoursuccessor)upforsomeverypainfulbughuntinglater.
ModernJVMscanreducethecostofincidentalsynchronizationbyoptimizingawaylockingthatcanbeprovenneverto contend. If a lock object is accessible only to the current thread, the JVM is permitted to optimize away a lock acquisition because there is no way another thread could synchronize on the same lock. For example, the lock acquisitioninListing11.2canalwaysbeeliminatedbytheJVM.
More sophisticated JVMs can use escape analysis to identify when a local object reference is never published to the heapandisthereforethreadͲlocal.IngetStoogeNamesinListing11.3,theonlyreferencetotheLististhelocalvariable stooges, and stackͲconfined variables are automatically threadͲlocal. A naive execution of getStoogeNames would acquireandreleasethelockontheVectorfourtimes,onceforeachcalltoaddortoString.However,asmartruntime compilercaninlinethesecallsandthenseethatstoogesanditsinternalstateneverescape,andthereforethatallfour lockacquisitionscanbeeliminated.[4]
[4]Thiscompileroptimization,calledlockelision,isperformedbytheIBMJVMandisexpectedinHotSpotasofJava7.
Listing11.3.CandidateforLockElision.
public String getStoogeNames() {
List<String> stooges = new Vector<String>();
stooges.add("Moe");
stooges.add("Larry");
stooges.add("Curly");
return stooges.toString();
}
Evenwithoutescapeanalysis,compilerscanalsoperformlockcoarsening,themergingofadjacentsynchronizedblocks usingthesamelock.ForgetStooge-Names,aJVMthatperformslockcoarseningmightcombinethethreecallstoadd and the call to toString into a single lock acquisition and release, using heuristics on the relative cost of synchronizationversustheinstructionsinsidethesynchronizedblock.[5]Notonlydoesthisreducethesynchronization overhead,butitalsogivestheoptimizeramuchlargerblocktoworkwith,likelyenablingotheroptimizations.
[5] A smart dynamic compiler can figure out that this method always returns the same string, and after the first execution recompile getStoogeNamestosimplyreturnthevaluereturnedbythefirstexecution.
Don'tworryexcessivelyaboutthecostofuncontendedsynchronization.Thebasicmechanismisalreadyquitefast,and JVMscanperformadditionaloptimizationsthatfurtherreduceoreliminatethecost.Instead,focusoptimizationefforts onareaswherelockcontentionactuallyoccurs.
Synchronizationbyonethreadcanalsoaffecttheperformanceofotherthreads.Synchronizationcreatestrafficonthe sharedmemorybus;thisbushasalimitedbandwidthandissharedacrossallprocessors.Ifthreadsmustcompetefor synchronizationbandwidth,allthreadsusingsynchronizationwillsuffer.[6]
[6] This aspect is sometimes used to argue against the use of nonͲblocking algorithms without some sort of backoff, because under heavy contention,nonͲblockingalgorithmsgeneratemoresynchronizationtrafficthanlockͲbasedones.SeeChapter15.
144 JavaConcurrencyInPractice
11.3.3.Blocking
UncontendedsynchronizationcanbehandledentirelywithintheJVM(Baconetal.,1998);contendedsynchronization mayrequireOSactivity,whichaddstothecost.Whenlockingiscontended,thelosingthread(s)mustblock.TheJVM
canimplementblockingeitherviaspinͲwaiting(repeatedlytryingtoacquirethelockuntilitsucceeds)orbysuspending theblockedthreadthroughtheoperatingsystem.Whichismoreefficientdependsontherelationshipbetweencontext switchoverheadandthetimeuntilthelockbecomesavailable;spinͲwaitingispreferableforshortwaitsandsuspension ispreferableforlongwaits.SomeJVMschoosebetweenthetwoadaptivelybasedonprofilingdataofpastwaittimes, butmostjustsuspendthreadswaitingforalock.
Suspendingathreadbecauseitcouldnotgetalock,orbecauseitblockedonaconditionwaitorblockingI/Ooperation, entailstwoadditionalcontextswitchesandalltheattendantOSandcacheactivity:theblockedthreadisswitchedout beforeitsquantumhasexpired,andisthenswitchedbackinlaterafterthelockorotherresourcebecomesavailable.
(Blockingduetolockcontentionalsohasacostforthethreadholdingthelock:whenitreleasesthelock,itmustthen asktheOStoresumetheblockedthread.)
11.4.ReducingLockContention
We've seen that serialization hurts scalability and that context switches hurt performance. Contended locking causes both,soreducinglockcontentioncanimprovebothperformanceandscalability.
Accesstoresourcesguardedbyanexclusivelockisserializedonlyonethreadatatimemayaccessit.Ofcourse,weuse locksforgoodreasons,suchaspreventingdatacorruption,butthissafetycomesataprice.Persistentcontentionfora locklimitsscalability.
Theprincipalthreattoscalabilityinconcurrentapplicationsistheexclusiveresourcelock.
Twofactorsinfluencethelikelihoodofcontentionforalock:howoftenthatlockisrequestedandhowlongitisheld once acquired.[7] If the product of these factors is sufficiently small, then most attempts to acquire the lock will be uncontended, and lock contention will not pose a significant scalability impediment. If, however, the lock is in sufficientlyhighdemand,threadswillblockwaitingforit;intheextremecase,processorswillsitidleeventhoughthere isplentyofworktodo.
[7]ThisisacorollaryofLittle'slaw,aresultfromqueuingtheorythatsays"theaveragenumberofcustomersinastablesystemisequaltotheir averagearrivalratemultipliedbytheiraveragetimeinthesystem".(Little,1961) Therearethreewaystoreducelockcontention:
x Reducethedurationforwhichlocksareheld;
x Reducethefrequencywithwhichlocksarerequested;or
x Replaceexclusivelockswithcoordinationmechanismsthatpermitgreaterconcurrency.
11.4.1.NarrowingLockScope("Getin,GetOut")
Aneffectivewaytoreducethelikelihoodofcontentionistoholdlocksasbrieflyaspossible.Thiscanbedonebymoving code that doesn't require the lock out of synchronized blocks, especially for expensive operations and potentially blockingoperationssuchasI/O.
It is easy to see how holding a "hot" lock for too long can limit scalability; we saw an example of this in SynchronizedFactorizerinChapter2.Ifanoperationholdsalockfor2millisecondsandeveryoperationrequiresthat lock, throughput can be no greater than 500 operations per second, no matter how many processors are available.
Reducingthetimethelockisheldto1millisecondimprovesthelockͲinducedthroughputlimitto1000operationsper second.[8]
[8]Actually,thiscalculationunderstatesthecostofholdinglocksfortoolongbecauseitdoesn'ttakeintoaccountthecontextswitchoverhead generatedbyincreasedlockcontention.
AttributeStoreinListing11.4showsanexampleofholdingalocklongerthannecessary.TheuserLocationMatches methodlooksuptheuser'slocationinaMapandusesregularexpressionmatchingtoseeiftheresultingvaluematches thesuppliedpattern.TheentireuserLocationMatchesmethodissynchronized,buttheonlyportionofthecodethat actuallyneedsthelockisthecalltoMap.get.
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 145
Listing11.4.HoldingaLockLongerthanNecessary.
@ThreadSafe
public class AttributeStore {
@GuardedBy("this") private final Map<String, String> attributes = new HashMap<String, String>();
public synchronized boolean userLocationMatches(String name,
String regexp) {
String key = "users." + name + ".location";
String location = attributes.get(key);
if (location == null)
return false;
else
return Pattern.matches(regexp, location);
}
}
BetterAttributeStoreinListing11.5rewritesAttributeStoretoreducesignificantlythelockduration.Thefirststep istoconstructtheMapkeyassociatedwiththeuser'slocation,astringoftheformusers.name.location.Thisentails instantiatingaStringBuilderobject,appendingseveralstringstoit,andinstantiatingtheresultasaString.Afterthe location has been retrieved, the regular expression is matched against the resulting location string. Because constructingthekeystringandprocessingtheregularexpressiondonotaccesssharedstate,theyneednotbeexecuted withthelockheld.BetterAttributeStorefactorsthesestepsoutofthesynchronizedblock,thusreducingthetime thelockisheld.
Listing11.5.ReducingLockDuration.
@ThreadSafe
public class BetterAttributeStore {
@GuardedBy("this") private final Map<String, String> attributes = new HashMap<String, String>();
public boolean userLocationMatches(String name, String regexp) {
String key = "users." + name + ".location";
String location;
synchronized (this) {
location = attributes.get(key);
}
if (location == null)
return false;
else
return Pattern.matches(regexp, location);
}
}
Reducing the scope of the lock in userLocationMatches substantially reduces the number of instructions that are executed with the lock held. By Amdahl's law, this removes an impediment to scalability because the amount of serializedcodeisreduced.
Because AttributeStore has only one state variable, attributes, we can improve it further by the technique of delegatingthreadsafety(Section4.3).ByreplacingattributeswithathreadͲsafeMap(aHashtable,synchronizedMap, or ConcurrentHashMap), AttributeStore can delegate all its thread safety obligations to the underlying threadͲsafe collection. This eliminates the need for explicit synchronization in AttributeStore, reduces the lock scope to the durationoftheMapaccess,andremovestheriskthatafuturemaintainerwillunderminethreadsafetybyforgettingto acquiretheappropriatelockbeforeaccessingattributes.
While shrinking synchronized blocks can improve scalability, a synchronized block can be too smalloperations that need to be atomic (such updating multiple variables that participate in an invariant) must be contained in a single synchronized block. And because the cost of synchronization is nonzero, breaking one synchronized block into multiple synchronized blocks (correctness permitting) at some point becomes counterproductive in terms of performance.[9]Theideal balanceisof courseplatformͲdependent,butinpracticeitmakessensetoworryaboutthe sizeofasynchronizedblockonlywhenyoucanmove"substantial"computationorblockingoperationsoutofit.
[9]IftheJVMperformslockcoarsening,itmayundothesplittingofsynchronizedblocksanyway.
11.4.2.ReducingLockGranularity
Theotherwaytoreducethefractionoftimethatalockisheld(andthereforethelikelihoodthatitwillbecontended)is to have threads ask for it less often. This can be accomplished by lock splitting and lock striping, which involve using
146 JavaConcurrencyInPractice
separate locks to guard multiple independent state variables previously guarded by a single lock. These techniques reducethegranularityatwhichlockingoccurs,potentiallyallowinggreaterscalabilitybutusingmorelocksalsoincreases theriskofdeadlock.
Asathoughtexperiment,imaginewhatwouldhappeniftherewasonlyonelockfortheentireapplicationinsteadofa separatelockforeachobject.Thenexecutionofallsynchronizedblocks,regardlessoftheirlock,wouldbeserialized.
Withmanythreadscompetingforthegloballock,thechancethattwothreadswantthelockatthesametimeincreases, resultinginmorecontention.Soiflockrequestswereinsteaddistributedoveralargersetoflocks,therewouldbeless contention.Fewerthreadswouldbeblockedwaitingforlocks,thusincreasingscalability.
If a lock guards more than one independent state variable, you may be able to improve scalability by splitting it into multiplelocksthateachguarddifferentvariables.Thisresultsineachlockbeingrequestedlessoften.
ServerStatusinListing11.6showsaportionofthemonitoringinterfaceforadatabaseserverthatmaintainsthesetof currentlyloggedͲonusersandthesetofcurrentlyexecutingqueries.Asauserlogsonorofforqueryexecutionbegins or ends, the ServerStatus object is updated by calling the appropriate add or remove method. The two types of information arecompletelyindependent; ServerStatuscould evenbesplitintotwoseparateclasses with nolossof functionality.
InsteadofguardingbothusersandquerieswiththeServerStatuslock,wecaninsteadguardeachwithaseparate lock,asshowninListing11.7.Aftersplittingthelock,eachnewfinerͲgrainedlockwillseelesslockingtrafficthanthe original coarser lock would have. (Delegating to a threadͲsafe Set implementation for users and queries instead of usingexplicitsynchronizationwouldimplicitlyprovidelocksplitting,aseachSetwoulduseadifferentlocktoguardits state.)
Splittingalockintotwooffersthegreatestpossibilityforimprovementwhenthelockisexperiencingmoderatebutnot heavycontention.Splittinglocksthatareexperiencinglittlecontentionyieldslittlenetimprovementinperformanceor throughput,althoughitmightincreasetheloadthresholdatwhichperformancestartstodegradeduetocontention.
Splittinglocksexperiencingmoderatecontentionmightactuallyturnthemintomostlyuncontendedlocks,whichisthe mostdesirableoutcomeforbothperformanceandscalability.
Listing11.6.CandidateforLockSplitting.
@ThreadSafe
public class ServerStatus {
@GuardedBy("this") public final Set<String> users;
@GuardedBy("this") public final Set<String> queries;
...
public synchronized void addUser(String u) { users.add(u); }
public synchronized void addQuery(String q) { queries.add(q); }
public synchronized void removeUser(String u) {
users.remove(u);
}
public synchronized void removeQuery(String q) {
queries.remove(q);
}
}
Listing11.7. ServerStatusRefactoredtoUseSplitLocks.
@ThreadSafe
public class ServerStatus {
@GuardedBy("users") public final Set<String> users;
@GuardedBy("queries") public final Set<String> queries;
...
public void addUser(String u) {
synchronized (users) {
users.add(u);
}
}
public void addQuery(String q) {
synchronized (queries) {
queries.add(q);
}
}
// remove methods similarly refactored to use split locks
}
11.4.3.LockStriping
Splittingaheavilycontendedlockintotwoislikelytoresultintwoheavilycontendedlocks.Whilethiswillproducea small scalability improvement by enabling two threads to execute concurrently instead of one, it still does not
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 147
dramatically improve prospects for concurrency on a system with many processors. The lock splitting example in the ServerStatusclassesdoesnotofferanyobviousopportunityforsplittingthelocksfurther.
Locksplittingcansometimesbeextendedtopartitionlockingonavariablesizedsetofindependentobjects,inwhich caseitiscalledlockstriping.Forexample,theimplementationofConcurrentHashMapusesanarrayof16locks,eachof which guards 1/16 of the hash buckets; bucket N is guarded by lock N mod 16. Assuming the hash function provides reasonablespreadingcharacteristicsandkeysareaccesseduniformly,thisshouldreducethedemandforanygivenlock byapproximatelyafactorof16.ItisthistechniquethatenablesConcurrentHashMaptosupportupto16concurrent writers. (The number of locks could be increased to provide even better concurrency under heavy access on highͲ
processorͲcountsystems,butthenumberofstripesshouldbeincreasedbeyondthedefaultof16onlywhenyouhave evidencethatconcurrentwritersaregeneratingenoughcontentiontowarrantraisingthelimit.) Oneofthedownsidesoflockstripingisthatlockingthecollectionforexclusiveaccessismoredifficultandcostlythan withasinglelock.Usuallyanoperationcanbeperformedbyacquiringatmostonelock,butoccasionallyyouneedto locktheentirecollection,aswhenConcurrentHashMapneedstoexpandthemapandrehashthevaluesintoalargerset ofbuckets.Thisistypicallydonebyacquiringallofthelocksinthestripeset.[10]
[10]Theonlywaytoacquireanarbitrarysetofintrinsiclocksisviarecursion.
StripedMapinListing11.8illustratesimplementingahashͲbasedmapusinglockstriping.ThereareN_LOCKSlocks,each guardingasubsetofthebuckets.Mostmethods,likeget,needacquireonlyasinglebucketlock.Somemethodsmay need to acquire all the locks but, as in the implementation for clear, may not need to acquire them all simultaneously.[11]
[11]ClearingtheMapinthiswayisnotatomic,sothereisnotnecessarilyatimewhentheStriped-Mapisactuallyemptyifotherthreadsare concurrentlyaddingelements;makingtheoperationatomicwouldrequireacquiringallthelocksatonce.However,forconcurrentcollectionsthat clientstypicallycannotlockforexclusiveaccess,theresultofmethodslikesizeorisEmptymaybeoutofdatebythetimetheyreturnanyway, sothisbehavior,whileperhapssomewhatsurprising,isusuallyacceptable.
11.4.4.AvoidingHotFields
Lock splitting and lock striping can improve scalability because they enable different threads to operate on different data (or different portions of the same data structure) without interfering with each other. A program that would benefitfromlocksplittingnecessarilyexhibitscontentionforalockmoreoftenthanforthedataguardedbythatlock.If alockguardstwoindependentvariablesXandY,andthreadAwantstoaccessXwhileBwantstoaccessY(aswouldbe thecaseifonethreadcalledaddUserwhileanothercalledaddQueryinServerStatus),thenthetwothreadsarenot contendingforanydata,eventhoughtheyarecontendingforalock.
148 JavaConcurrencyInPractice
Listing11.8.HashǦbasedMapUsingLockStriping.
@ThreadSafe
public class StripedMap {
// Synchronization policy: buckets[n] guarded by locks[n%N_LOCKS]
private static final int N_LOCKS = 16;
private final Node[] buckets;
private final Object[] locks;
private static class Node { ... }
public StripedMap(int numBuckets) {
buckets = new Node[numBuckets];
locks = new Object[N_LOCKS];
for (int i = 0; i < N_LOCKS; i++)
locks[i] = new Object();
}
private final int hash(Object key) {
return Math.abs(key.hashCode() % buckets.length);
}
public Object get(Object key) {
int hash = hash(key);
synchronized (locks[hash % N_LOCKS]) {
for (Node m = buckets[hash]; m != null; m = m.next)
if (m.key.equals(key))
return m.value;
}
return null;
}
public void clear() {
for (int i = 0; i < buckets.length; i++) {
synchronized (locks[i % N_LOCKS]) {
buckets[i] = null;
}
}
}
...
}
Lockgranularitycannotbereducedwhentherearevariablesthatarerequiredforeveryoperation.Thisisyetanother areawhererawperformanceandscalabilityareoftenatoddswitheachother;commonoptimizationssuchascaching frequentlycomputedvaluescanintroduce"hotfields"thatlimitscalability.
IfyouwereimplementingHashMap,youwouldhaveachoiceofhowsizecomputesthenumberofentriesintheMap.
Thesimplestapproachistocountthenumberofentrieseverytimeitiscalled.Acommonoptimizationistoupdatea separatecounterasentriesareaddedorremoved;thisslightlyincreasesthecostofaputorremoveoperationtokeep thecounterupͲtoͲdate,butreducesthecostofthesizeoperationfromO(n)toO(1).
Keeping a separate count to speed up operations like size and isEmpty works fine for a singleͲthreaded or fully synchronized implementation, but makes it much harder to improve the scalability of the implementation because everyoperationthatmodifiesthemapmustnowupdatethesharedcounter.Evenifyouuselockstripingforthehash chains,synchronizingaccesstothecounterreintroducesthescalabilityproblemsofexclusivelocking.Whatlookedlikea performanceoptimizationͲcachingtheresultsofthesizeoperationͲhasturnedintoascalabilityliability.Inthiscase, thecounteriscalledahotfieldbecauseeverymutativeoperationneedstoaccessit.
ConcurrentHashMapavoidsthisproblembyhavingsizeenumeratethestripesandaddupthenumberofelementsin eachstripe,insteadofmaintainingaglobalcount.Toavoidenumeratingeveryelement,ConcurrentHashMapmaintains aseparatecountfieldforeachstripe,alsoguardedbythestripelock.[12]
[12]Ifsizeiscalledfrequentlycomparedtomutativeoperations,stripeddatastructurescanoptimizeforthisbycachingthecollectionsizeina volatile whenever size is called and invalidating the cache (setting it to Ͳ1) whenever the collection is modified. If the cached value is nonnegativeonentrytosize,itisaccurateandcanbereturned;otherwiseitisrecomputed.
11.4.5.AlternativestoExclusiveLocks
Athirdtechniqueformitigatingtheeffectoflockcontentionistoforegotheuseofexclusivelocksinfavorofamore concurrencyͲfriendlymeansofmanagingsharedstate.Theseincludeusingtheconcurrentcollections,readͲwritelocks, immutableobjectsandatomicvariables.
ReadWriteLock(seeChapter13)enforcesamultipleͲreader,singleͲwriterlockingdiscipline:morethanonereadercan accessthesharedresourceconcurrentlysolongasnoneofthemwantstomodifyit,butwritersmustacquirethelock
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 149
exclusively. For readͲmostly data structures,ReadWriteLock can offer greater concurrency than exclusive locking; for readͲonlydatastructures,immutabilitycaneliminatetheneedforlockingentirely.
Atomicvariables(seeChapter15)offerameansofreducingthecostofupdating"hotfields"suchasstatisticscounters, sequencegenerators,orthereferencetothefirstnodeinalinkeddatastructure.(WeusedAtomicLongtomaintainthe hitcounterintheservletexamplesinChapter2.)TheatomicvariableclassesprovideveryfineͲgrained(andtherefore more scalable) atomic operations on integers or object references, and areimplemented using lowͲlevel concurrency primitives (suchascompareͲandͲswap) providedby mostmodernprocessors.Ifyourclasshasasmallnumberofhot fields that do not participate in invariants with other variables, replacing them with atomic variables may improve scalability. (Changing your algorithm to have fewer hot fields might improve scalability even more Ͳ atomic variables reducethecostofupdatinghotfields,buttheydon'teliminateit.)
11.4.6.MonitoringCPUUtilization
When testing for scalability, the goal is usually to keep the processors fully utilized. Tools like vmstat and mpstat on UnixsystemsorperfmononWindowssystemscantellyoujusthow"hot"theprocessorsarerunning.
IftheCPUsareasymmetricallyutilized(someCPUsarerunninghotbutothersarenot)yourfirstgoalshouldbetofind increasedparallelisminyourprogram.Asymmetricutilizationindicatesthatmostofthecomputation isgoingonina smallsetofthreads,andyourapplicationwillnotbeabletotakeadvantageofadditionalprocessors.
IftheCPUsarenotfullyutilized,youneedtofigureoutwhy.Thereareseverallikelycauses: Insufficientload.Itmaybethattheapplicationbeingtestedisjustnotsubjectedtoenoughload.Youcantestforthisby increasing the load and measuring changes in utilization, response time, or service time. Generating enough load to saturateanapplicationcanrequiresubstantialcomputerpower;theproblemmaybethattheclientsystems,notthe systembeingtested,arerunningatcapacity.
I/OͲbound. You can determine whether an application is diskͲbound using iostat or perfmon, and whether it is bandwidthͲlimitedbymonitoringtrafficlevelsonyournetwork.
Externally bound. If your application depends on external services such as a database or web service, the bottleneck maynotbeinyourcode. Youcantest forthisbyusingaprofilerordatabaseadministrationtoolstodeterminehow muchtimeisbeingspentwaitingforanswersfromtheexternalservice.
Lockcontention.Profilingtoolscantellyouhowmuchlockcontentionyourapplicationisexperiencingandwhichlocks are"hot".Youcanoftengetthesameinformationwithoutaprofilerthroughrandomsampling,triggeringafewthread dumpsandlookingforthreadscontendingforlocks.Ifathreadisblockedwaitingforalock,theappropriatestackframe in the thread dump indicates "waiting to lock monitor . . . " Locks that are mostly uncontended rarely show up in a thread dump; a heavily contended lock will almost always have at least one thread waiting to acquire it and so will frequentlyappearinthreaddumps.
IfyourapplicationiskeepingtheCPUssufficientlyhot,youcanusemonitoringtoolstoinferwhetheritwouldbenefit fromadditionalCPUs.Aprogramwithonlyfourthreadsmaybeabletokeepa4Ͳwaysystemfullyutilized,butisunlikely toseeaperformanceboostifmovedtoan8Ͳwaysystemsincetherewouldneedtobewaitingrunnablethreadstotake advantageoftheadditionalprocessors.(Youmayalsobeabletoreconfiguretheprogramtodivideitsworkloadover morethreads,suchasadjustingathreadpoolsize.)Oneofthecolumnsreportedbyvmstatisthenumberofthreads thatarerunnablebutnotcurrentlyrunningbecauseaCPUisnotavailable;ifCPUutilizationishighandtherearealways runnablethreadswaitingforaCPU,yourapplicationwouldprobablybenefitfrommoreprocessors.
11.4.7.JustSayNotoObjectPooling
In early JVM versions, object allocation and garbage collection were slow,[13] but their performance has improved substantially since then. In fact, allocation in Java is now faster than malloc is in C: the common code path for new ObjectinHotSpot1.4.xand5.0isapproximatelytenmachineinstructions.
[13]AswaseverythingelseͲsynchronization,graphics,JVMstartup,reflectionͲpredictablysointhefirstversionofanexperimentaltechnology.
Toworkaround"slow"objectlifecycles,manydevelopersturnedtoobjectpooling,whereobjectsarerecycledinstead ofbeinggarbagecollectedandallocatedanewwhenneeded.Eventakingintoaccountitsreducedgarbagecollection overhead, object pooling has been shown to be a performance loss[14] for all but the most expensive objects (and a seriouslossforlightͲandmediumͲweightobjects)insingleͲthreadedprograms(Click,2005).
150 JavaConcurrencyInPractice
[14]InadditiontobeingalossintermsofCPUcycles,objectpoolinghasanumberofotherproblems,amongthemthechallengeofsettingpool sizescorrectly(toosmall,andpoolinghasnoeffect;toolarge,anditputspressureonthegarbagecollector,retainingmemorythatcouldbeused moreeffectivelyforsomethingelse);theriskthatanobjectwillnotbeproperlyresettoitsnewlyallocatedstate,introducingsubtlebugs;therisk that a thread will return an object to the pool but continue using it; and that it makes more work for generational garbage collectors by encouragingapatternofoldͲtoͲyoungreferences.
In concurrent applications, pooling fares even worse. When threads allocate new objects, very little interͲthread coordinationisrequired,asallocatorstypicallyusethreadͲlocalallocationblockstoeliminatemostsynchronizationon heapdatastructures.Butifthosethreadsinsteadrequestanobjectfromapool,somesynchronizationisnecessaryto coordinateaccesstothepooldatastructure,creatingthepossibilitythatathreadwillblock.Becauseblockingathread duetolockcontentionishundredsoftimesmoreexpensivethananallocation,evenasmallamountofpoolͲinduced contention would be a scalability bottleneck. (Even an uncontended synchronization is usually more expensive than allocating an object.) This is yet another technique intended as a performance optimization but that turned into a scalabilityhazard.Poolinghasitsuses,[15]butisoflimitedutilityasaperformanceoptimization.
[15]Inconstrainedenvironments,suchassomeJ2MEorRTSJtargets,objectpoolingmaystillberequiredforeffectivememorymanagementorto manageresponsiveness.
Allocatingobjectsisusuallycheaperthansynchronizing.
11.5.Example:ComparingMapPerformance
ThesingleͲthreadedperformanceofConcurrentHashMapisslightlybetterthanthatofasynchronizedHashMap,butitis in concurrent use that it really shines. The implementation of ConcurrentHashMap assumes the most common operation is retrieving a value that already exists, and is therefore optimized to provide highest performance and concurrencyforsuccessfulgetoperations.
ThemajorscalabilityimpedimentforthesynchronizedMapimplementationsisthatthereisasinglelockfortheentire map,soonlyonethreadcanaccessthemapatatime.Ontheotherhand,ConcurrentHashMapdoesnolockingformost successfulreadoperations,anduseslockstripingforwriteoperationsandthosefewreadoperationsthatdorequire locking.Asaresult,multiplethreadscanaccesstheMapconcurrentlywithoutblocking.
Figure 11.3 illustrates the differences in scalability between several Map implementations: ConcurrentHashMap, ConcurrentSkipListMap,andHashMapandtreeMapwrappedwithsynchronizedMap.ThefirsttwoarethreadͲsafeby design;thelattertwoaremadethreadͲsafebythesynchronizedwrapper.Ineachrun,Nthreadsconcurrentlyexecutea tightloopthatselectsarandomkeyandattempts toretrievethevaluecorrespondingtothatkey.Ifthevalueisnot present,itisaddedtotheMapwithprobabilityp=.6,andifitispresent,isremovedwithprobabilityp=.02.Thetests wererununderapreͲreleasebuildofJava6onan8ͲwaySparcV880,andthegraphdisplaysthroughputnormalizedto theonethreadcaseforConcurrentHashMap.(Thescalabilitygapbetweentheconcurrentandsynchronizedcollectionsis evenlargeronJava5.0.)
ThedataforConcurrentHashMapandConcurrentSkipListMapshowsthattheyscalewelltolargenumbersofthreads; throughput continues to improve as threads are added. While the numbers of threads in Figure 11.3 may not seem large,thistestprogramgeneratesmorecontentionperthreadthanatypicalapplicationbecauseitdoeslittleotherthan poundontheMap;arealprogramwoulddoadditionalthreadͲlocalworkineachiteration.
Figure11.3.ComparingScalabilityofMapImplementations.
[Viewfullsizeimage]
6BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability 151
The numbers for the synchronized collections are not as encouraging. Performance for the oneͲthread case is comparable to ConcurrentHashMap, but once the load transitions from mostly uncontended to mostly contended Ͳ
whichhappenshereattwothreadsͲthesynchronizedcollectionssufferbadly.Thisiscommonbehaviorforcodewhose scalabilityislimitedbylockcontention.Solongascontentionislow,timeperoperationisdominatedbythetimeto actuallydotheworkandthroughputmayimproveasthreadsareadded.Oncecontentionbecomessignificant,timeper operation is dominated by context switch and scheduling delays, and adding more threads has little effect on throughput.
11.6.ReducingContextSwitchOverhead
Manytasksinvolveoperationsthatmayblock;transitioningbetweentherunningandblockedstatesentailsacontext switch.Onesourceofblockinginserverapplicationsisgeneratinglogmessagesinthecourseofprocessingrequests;to illustratehowthroughputcanbeimprovedbyreducingcontextswitches,we'llanalyzetheschedulingbehavioroftwo loggingapproaches.
Mostloggingframeworksarethinwrappersaround println;whenyouhavesomething tolog,justwriteitoutright then and there. Another approach was shown in LogWriter on page 152: the logging is performed in a dedicated backgroundthreadinsteadofbytherequestingthread.Fromthedeveloper'sperspective,bothapproachesareroughly equivalent. But there may be a difference in performance, depending on the volume of logging activity, how many threadsaredoinglogging,andotherfactorssuchasthecostofcontextswitching.[16]
[16]BuildingaloggerthatmovestheI/Otoanotherthreadmayimproveperformance,butitalsointroducesanumberofdesigncomplications, suchasinterruption(whathappensifathreadblockedinaloggingoperationisinterrupted?),serviceguarantees(doestheloggerguaranteethata successfullyqueuedlogmessagewillbeloggedpriortoserviceshutdown?),saturationpolicy(whathappenswhentheproducerslogmessages fasterthantheloggerthreadcanhandlethem?),andservicelifecycle(howdoweshutdownthelogger,andhowdowecommunicatetheservice statetoproducers?).
TheservicetimeforaloggingoperationincludeswhatevercomputationisassociatedwiththeI/Ostreamclasses;ifthe I/Ooperationblocks,italsoincludesthedurationforwhichthethreadisblocked.Theoperatingsystemwilldeschedule the blocked thread until the I/O completes, and probably a little longer. When the I/O completes, other threads are probablyactiveandwillbeallowedtofinishouttheirschedulingquanta,andthreadsmayalreadybewaitingaheadof usontheschedulingqueueͲfurtheraddingtoservicetime.Alternatively,ifmultiplethreadsareloggingsimultaneously, there may be contention for the output stream lock, in which case the result is the same as with blocking I/O Ͳ the thread blocks waiting for the lock and gets switched out. Inline logging involves I/O and locking, which can lead to increasedcontextswitchingandthereforeincreasedservicetimes.
Increasing request service time is undesirable for several reasons. First, service time affects quality of service: longer servicetimesmeansomeoneiswaitinglongerforaresult.Butmoresignificantly,longerservicetimesinthiscasemean more lock contention. The "get in, get out" principle of Section 11.4.1 tells us that we should hold locks as briefly as possible,becausethelongeralockisheld,themorelikelythatlockwillbecontended.IfathreadblockswaitingforI/O
while holding a lock, another thread is more likely to want the lock while the first thread is holding it. Concurrent systemsperformmuchbetterwhenmostlockacquisitionsareuncontended,becausecontendedlockacquisitionmeans morecontextswitches.Acodingstylethatencouragesmorecontextswitchesthusyieldsloweroverallthroughput.
MovingtheI/OoutoftherequestͲprocessingthreadislikelytoshortenthemeanservicetimeforrequestprocessing.
ThreadscallinglognolongerblockwaitingfortheoutputstreamlockorforI/Otocomplete;theyneedonlyqueuethe messageandcanthenreturntotheir task.On the otherhand,we'veintroducedthepossibilityofcontentionforthe messagequeue,buttheputoperationislighterͲweightthantheloggingI/O(whichmightrequiresystemcalls)andsois lesslikelytoblockinactualuse(aslongasthequeueisnotfull).Becausetherequestthreadisnowlesslikelytoblock,it is less likely to be contextͲswitched out in the middle of a request. What we've done is turned a complicated and uncertaincodepathinvolvingI/OandpossiblelockcontentionintoastraightͲlinecodepath.
Tosomeextent,wearejustmovingtheworkaround,movingtheI/Otoathreadwhereitscostisn'tperceivedbythe user(whichmayinitselfbeawin).ButbymovingalltheloggingI/Otoasinglethread,wealsoeliminatethechanceof contention for the output stream and thus eliminate a source of blocking. This improves overall throughput because fewerresourcesareconsumedinscheduling,contextswitching,andlockmanagement.
MovingtheI/OfrommanyrequestͲprocessingthreadstoasingleloggerthreadissimilartothedifferencebetweena bucket brigade and a collection of individuals fighting a fire. In the "hundred guys running around with buckets"
approach,youhaveagreaterchanceofcontentionatthewatersourceandatthefire(resultinginoveralllesswater
152 JavaConcurrencyInPractice
delivered to the fire), plus greater inefficiency because each worker is continuously switching modes (filling, running, dumping,running,etc.).InthebucketͲbrigadeapproach,theflowofwaterfromthesourcetotheburningbuildingis constant, less energy is expended transporting the water to the fire, and each worker focuses on doing one job continuously.JustasinterruptionsaredisruptiveandproductivityͲreducingtohumans,blockingandcontextswitching aredisruptivetothreads.
Summary
Because one of the most common reasons to use threads is to exploit multiple processors, in discussing the performanceofconcurrentapplications,weareusuallymoreconcernedwiththroughputorscalabilitythanwearewith rawservicetime.Amdahl'slawtellsusthatthescalabilityofanapplicationisdrivenbytheproportionofcodethatmust beexecutedserially.SincetheprimarysourceofserializationinJavaprogramsistheexclusiveresourcelock,scalability canoftenbeimprovedbyspendinglesstimeholdinglocks,eitherbyreducinglockgranularity,reducingthedurationfor whichlocksareheld,orreplacingexclusivelockswithnonexclusiveornonͲblockingalternatives.
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 153
Chapter12.TestingConcurrentPrograms
Concurrent programs employ similar design principles and patterns to sequential programs. The difference is that concurrent programs have a degree of nonͲdeterminism that sequential programs do not, increasing the number of potentialinteractionsandfailuremodesthatmustbeplannedforandanalyzed.
Similarly, testing concurrent programs uses and extends ideas from testing sequential ones. The same techniques for testing correctness and performance in sequential programs can be applied to concurrent programs, but with concurrentprogramsthespaceofthingsthatcangowrongismuchlarger.Themajorchallengeinconstructingtestsfor concurrent programs is that potential failures may be rare probabilistic occurrences rather than deterministic ones; teststhatdisclosesuchfailuresmustbemoreextensiveandrunforlongerthantypicalsequentialtests.
Mosttestsofconcurrentclassesfallintooneorbothoftheclassiccategoriesofsafetyandliveness.InChapter1,we definedsafetyas"nothingbadeverhappens"andlivenessas"somethinggoodeventuallyhappens".
Tests of safety, which verify that a class's behavior conforms to its specification, usually take the form of testing invariants. For example, in a linked list implementation that caches the size of the list every time it is modified, one safetytestwouldbetocomparethecachedcountagainsttheactualnumberofelementsinthelist.InasingleͲthreaded program this is easy, since the list contents do not change while you are testing its properties. But in a concurrent program,suchatestislikelytobefraughtwithracesunlessyoucanobservethecountfieldandcounttheelementsina single atomic operation. This can be done by locking the list for exclusive access, employing some sort of "atomic snapshot"featureprovidedbytheimplementation,orbyusing"testpoints"providedbytheimplementationthatlet testsassertinvariantsorexecutetestcodeatomically.
In this book, we've used timing diagrams to depict "unlucky" interactions that could cause failures in incorrectly constructedclasses;testprogramsattempttosearchenoughofthestatespacethatsuchbadluckeventuallyoccurs.
Unfortunately, test code can introduce timing or synchronization artifacts that can mask bugs that might otherwise manifestthemselves.[1]
[1]BugsthatdisappearwhenyouadddebuggingortestcodeareplayfullycalledHeisenbugs.
Liveness properties present their own testing challenges. Liveness tests include tests of progress and nonͲprogress, whicharehardtoquantifyͲhowdoyouverifythatamethodisblockingandnotmerelyrunningslowly?Similarly,how doyoutestthatanalgorithmdoesnotdeadlock?Howlongshouldyouwaitbeforeyoudeclareittohavefailed?
Relatedtolivenesstestsareperformancetests.Performancecanbemeasuredinanumberofways,including: Throughput:therateatwhichasetofconcurrenttasksiscompleted;
Responsiveness:thedelaybetweenarequestforandcompletionofsomeaction(alsocalledlatency);or Scalability:theimprovementinthroughput(orlackthereof)asmoreresources(usuallyCPUs)aremadeavailable.
12.1.TestingforCorrectness
DevelopingunittestsforaconcurrentclassstartswiththesameanalysisasforasequentialclassͲidentifyinginvariants and postͲconditions that are amenable to mechanical checking. If you are lucky, many of these are present in the specification;therestofthetime,writingtestsisanadventureiniterativespecificationdiscovery.
As a concrete illustration, we're going to build a set of test cases for a bounded buffer. Listing 12.1 shows our BoundedBufferimplementation,usingSemaphoretoimplementtherequiredboundingandblocking.
BoundedBufferimplementsafixedͲlengtharrayͲbasedqueuewithblockingputandtakemethodscontrolledbyapair of counting semaphores. The availableItems semaphore represents the number of elements that can be removed from the buffer, and is initially zero (since the buffer is initially empty). Similarly, availableSpaces represents how manyitemscanbeinsertedintothebuffer,andisinitializedtothesizeofthebuffer.
AtakeoperationfirstrequiresthatapermitbeobtainedfromavailableItems.Thissucceedsimmediatelyifthebuffer is nonempty, and otherwise blocks until the buffer becomes nonempty. Once a permit is obtained, the next element fromthebufferisremovedandapermitisreleasedtotheavailableSpacessemaphore.[2]Theputoperationisdefined conversely, so that on exit from either the put or take methods, the sum of the counts of both semaphores always equals the bound. (In practice, if you need a bounded buffer you should use ArrayBlockingQueue or LinkedBlockingQueue rather than rolling your own, but the technique used here illustrates how insertions and removalscanbecontrolledinotherdatastructuresaswell.)
154 JavaConcurrencyInPractice
[2] In a counting semaphore, the permits are not represented explicitly or associated with an owning thread; a release operation creates a permitandanacquireoperationconsumesone.
Listing12.1.BoundedBufferUsingSemaphore.
@ThreadSafe
public class BoundedBuffer<E> {
private final Semaphore availableItems, availableSpaces;
@GuardedBy("this") private final E[] items;
@GuardedBy("this") private int putPosition = 0, takePosition = 0; public BoundedBuffer(int capacity) {
availableItems = new Semaphore(0);
availableSpaces = new Semaphore(capacity);
items = (E[]) new Object[capacity];
}
public boolean isEmpty() {
return availableItems.availablePermits() == 0;
}
public boolean isFull() {
return availableSpaces.availablePermits() == 0;
}
public void put(E x) throws InterruptedException {
availableSpaces.acquire();
doInsert(x);
availableItems.release();
}
public E take() throws InterruptedException {
availableItems.acquire();
E item = doExtract();
availableSpaces.release();
return item;
}
private synchronized void doInsert(E x) {
int i = putPosition;
items[i] = x;
putPosition = (++i == items.length)? 0 : i;
}
private synchronized E doExtract() {
int i = takePosition;
E x = items[i];
items[i] = null;
takePosition = (++i == items.length)? 0 : i;
return x;
}
}
12.1.1.BasicUnitTests
The most basic unit tests for BoundedBuffer are similar to what we'd use in a sequential contextcreate a bounded buffer,callitsmethods,andassertpostͲconditionsandinvariants.Someinvariantsthatquicklycometomindarethata freshlycreatedbuffershouldidentifyitselfasempty,andalsoasnotfull.Asimilarbutslightlymorecomplicatedsafety test is to insert N elements into a buffer with capacity N (which should succeed without blocking), and test that the bufferrecognizesthatitisfull(andnotempty).JUnittestmethodsforthesepropertiesareshowninListing12.2.
Listing12.2.BasicUnitTestsforBoundedBuffer.
class BoundedBufferTest extends TestCase {
void testIsEmptyWhenConstructed() {
BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10); assertTrue(bb.isEmpty());
assertFalse(bb.isFull());
}
void testIsFullAfterPuts() throws InterruptedException {
BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10); for (int i = 0; i < 10; i++)
bb.put(i);
assertTrue(bb.isFull());
assertFalse(bb.isEmpty());
}
}
Thesesimpletestmethodsareentirelysequential.Includingasetofsequentialtestsinyourtestsuiteisoftenhelpful, sincetheycandisclosewhenaproblemisnotrelatedtoconcurrencyissuesbeforeyoustartlookingfordataraces.
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 155
12.1.2.TestingBlockingOperations
Testsofessentialconcurrencypropertiesrequireintroducingmorethanonethread.Mosttestingframeworksarenot veryconcurrencyͲfriendly:theyrarelyincludefacilitiestocreatethreadsormonitorthemtoensurethattheydonotdie unexpectedly.Ifahelperthreadcreatedbyatestcasediscoversafailure,theframeworkusuallydoesnotknowwith whichtestthethreadisassociated,sosomeworkmayberequiredtorelaysuccessorfailureinformationbacktothe maintestrunnerthreadsoitcanbereported.
For the conformance tests for java.util.concurrent, it was important that failures be clearly associated with a specifictest.HencetheJSR166ExpertGroupcreatedabaseclass[3]thatprovidedmethodstorelayandreportfailures duringtearDown,followingtheconventionthateverytestmustwaituntilallthethreadsitcreatedterminate.Youmay not need to go to such lengths; the key requirements are that it be clear whether the tests passed and that failure informationisreportedsomewhereforuseindiagnosingtheproblem.
[3]http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/tck/JSR166TestCase.java If a method is supposed to block under certain conditions, then a test for that behavior should succeed only if the threaddoesnotproceed.Testingthatamethodblocksissimilartotestingthatamethodthrowsanexception;ifthe methodreturnsnormally,thetesthasfailed.
Testingthatamethodblocksintroducesanadditionalcomplication:oncethemethodsuccessfullyblocks,youhaveto convince it somehow to unblock. The obvious way to do this is via interruptionstart a blocking activity in a separate thread,waituntilthethreadblocks,interruptit,andthenassertthattheblockingoperationcompleted.Ofcourse,this requiresyourblockingmethodstorespondtointerruptionbyreturningearlyorthrowingInterruptedException.
The "wait until the thread blocks" part is easier said than done; in practice, you have to make an arbitrary decision about how long the few instructions being executed could possibly take, and wait longer than that. You should be preparedtoincreasethisvalueifyouarewrong(inwhichcaseyouwillseespurioustestfailures).
Listing 12.3 shows an approach to testing blocking operations. It creates a "taker" thread that attempts to take an elementfromanemptybuffer.Iftakesucceeds,itregistersfailure.Thetestrunnerthreadstartsthetakerthread,waits a long time, and then interrupts it. If the taker thread has correctly blocked in the take operation, it will throw InterruptedException,andthecatchblockforthisexceptiontreatsthisassuccessandletsthethreadexit.Themain testrunnerthreadthenattemptstojoinwiththetakerthreadandverifiesthatthejoinreturnedsuccessfullybycalling Thread.isAlive;ifthetakerthreadrespondedtotheinterrupt,thejoinshouldcompletequickly.
Thetimedjoinensuresthatthetestcompleteseveniftakegetsstuckinsomeunexpectedway.Thistestmethodtests severalpropertiesoftakenotonlythatitblocksbutthat,wheninterrupted,itthrowsInterruptedException.Thisis one of the few cases in which it is appropriate to subclass Thread explicitly instead of using a Runnable in a pool: in ordertotestproperterminationwithjoin.Thesameapproachcanbeusedtotestthatthetakerthreadunblocksafter anelementisplacedinthequeuebythemainthread.
It is tempting to use Thread.getState to verify that the thread is actually blocked on a condition wait, but this approachisnotreliable.ThereisnothingthatrequiresablockedthreadevertoentertheWAITINGorTIMED_WAITING
states,since the JVMcan choosetoimplement blocking byspinͲwaitinginstead.Similarly,becausespuriouswakeups from Object.wait or Condition.await are permitted (see Chapter 14), a thread in the WAITING or TIMED_WAITING
statemaytemporarilytransitiontoRUNNABLEeveniftheconditionforwhichitiswaitingisnotyettrue.Evenignoring theseimplementationoptions,itmaytakesometimeforthetargetthreadtosettleintoablockingstate.Theresultof Thread.getStateshouldnotbeusedforconcurrencycontrol,andisoflimitedusefulnessfortestingͲitsprimaryutility isasasourceofdebugginginformation.
156 JavaConcurrencyInPractice
Listing12.3.TestingBlockingandResponsivenesstoInterruption.
void testTakeBlocksWhenEmpty() {
final BoundedBuffer<Integer> bb = new BoundedBuffer<Integer>(10); Thread taker = new Thread() {
public void run() {
try {
int unused = bb.take();
fail(); // if we get here, it's an error
} catch (InterruptedException success) { }
}};
try {
taker.start();
Thread.sleep(LOCKUP_DETECT_TIMEOUT);
taker.interrupt();
taker.join(LOCKUP_DETECT_TIMEOUT);
assertFalse(taker.isAlive());
} catch (Exception unexpected) {
fail();
}
}
12.1.3.TestingSafety
ThetestsinListings12.2and12.3testimportantpropertiesoftheboundedbuffer,butareunlikelytodiscloseerrors stemmingfromdataraces.Totestthataconcurrentclassperformscorrectlyunderunpredictableconcurrentaccess,we needtosetupmultiplethreadsperformingputandtakeoperationsoversomeamountoftimeandthensomehowtest thatnothingwentwrong.
Constructing tests to disclose safety errors in concurrent classes is a chickenͲandͲegg problem: the test programs themselves are concurrent programs. Developing good concurrent tests can be more difficult than developing the classestheytest.
Thechallengetoconstructingeffectivesafetytestsforconcurrentclassesisidentifyingeasilycheckedpropertiesthat will,withhighprobability,failifsomethinggoeswrong,whileatthesametimenotlettingthefailureͲauditingcodelimit concurrencyartificially.Itisbestifcheckingthetestpropertydoesnotrequireanysynchronization.
One approach that works well with classes used in producerͲconsumer designs (like BoundedBuffer) is to check that everything put into a queue or buffer comes out of it, and that nothing else does. A naive implementation of this approachwouldinserttheelementintoa"shadow"listwhenitisputonthequeue,removeitfromthelistwhenitis removedfromthequeue,andassertthattheshadowlistisemptywhenthetesthasfinished.Butthisapproachwould distorttheschedulingofthetestthreadsbecausemodifyingtheshadowlistwouldrequiresynchronizationandpossibly blocking.
AbetterapproachistocomputechecksumsoftheelementsthatareenqueuedanddequeuedusinganorderͲsensitive checksumfunction,andcomparethem.Iftheymatch,thetestpasses.Thisapproachworksbestwhenthereisasingle producerputtingelementsintothebufferandasingleconsumertakingthemout,becauseitcantestnotonlythatthe rightelements(probably)cameoutbutthattheycameoutintherightorder.
ExtendingthisapproachtoamultipleͲproducer,multipleͲconsumersituationrequiresusingachecksumfunctionthatis insensitive to the order in which the elements are combined, so that multiple checksums can be combined after the test.Otherwise,synchronizingaccesstoasharedchecksumfieldcouldbecomeaconcurrencybottleneckordistortthe timingofthetest.(Anycommutativeoperation,suchasadditionorXOR,meetstheserequirements.) To ensure that your test actually tests what you think it does, it is important that the checksums themselves not be guessablebythecompiler.Itwouldbeabadideatouseconsecutiveintegersasyourtestdatabecausethentheresult wouldalwaysbethesame,andasmartcompilercouldconceivablyjustprecomputeit.
Toavoidthisproblem,testdatashouldbegeneratedrandomly,butmanyotherwiseeffectivetestsarecompromisedby apoorchoiceofrandomnumbergenerator(RNG).Randomnumbergenerationcancreatecouplingsbetweenclasses andtimingartifactsbecausemostrandomnumbergeneratorclassesarethreadsafeandthereforeintroduceadditional synchronization.[4]GivingeachthreaditsownRNGallowsanonͲthreadͲsafeRNGtobeused.
[4]Manybenchmarksare,unbeknownsttotheirdevelopersorusers,simplytestsofhowgreataconcurrencybottlenecktheRNGis.
Rather than using a generalͲpurpose RNG, it is better to use simple pseudorandom functions. You don't need highͲ
qualityrandomness;allyouneedisenoughrandomnesstoensurethenumberschangefromruntorun.Thexor-Shift functioninListing12.4(Marsaglia,2003)isamongthecheapestmediumͲqualityrandomnumberfunctions.Startingit
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 157
off with values based on hashCode and nanoTime makes the sums both unguessable and almost always different for eachrun.
Listing12.4.MediumǦqualityRandomNumberGeneratorSuitableforTesting.
static int xorShift(int y) {
y ^= (y << 6);
y ^= (y >>> 21);
y ^= (y << 7);
return y;
}
PutTakeTest in Listings 12.5 and 12.6 starts N producer threads that generate elements and enqueue them, and N
consumerthreadsthatdequeuethem.Eachthreadupdatesthechecksumoftheelementsastheygoinorout,usinga perthreadchecksumthatiscombinedattheendofthetestrunsoastoaddnomoresynchronizationorcontention thanrequiredtotestthebuffer.
Dependingonyourplatform,creatingandstartingathreadcanbeamoderatelyheavyweightoperation.Ifyourthread isshortͲrunningandyoustartanumberofthreadsinaloop,thethreadsrunsequentiallyratherthanconcurrentlyin theworstcase.EveninthenotͲquiteͲworstcase,thefactthatthefirstthreadhasaheadstartontheothersmeansthat youmaygetfewerinterleavingsthanexpected:thefirstthreadrunsbyitselfforsomeamountoftime,andthenthe first two threads run concurrently for some amount of time, and only eventually are all the threads running concurrently.(Thesamethinghappensattheendoftherun:thethreadsthatgotaheadstartalsofinishearly.) WepresentedatechniqueformitigatingthisprobleminSection5.5.1,usingaCountDownLatchasastartinggateand anotherasafinishgate.AnotherwaytogetthesameeffectistouseaCyclicBarrier,initializedwiththenumberof workerthreadsplusone,andhavetheworkerthreadsandthetestdriverwaitatthebarrieratthebeginningandendof theirrun.Thisensuresthatallthreadsareupandrunningbeforeanystartworking.PutTakeTestusesthistechniqueto coordinate starting and stopping the worker threads, creating more potential concurrent interleavings. We still can't guarantee that the scheduler won't run each thread to completion sequentially, but making the runs long enough reducestheextenttowhichschedulingdistortsourresults.
The final trick employed by PutTakeTest is to use a deterministic termination criterion so that no additional interͲ
threadcoordinationisneededtofigureoutwhenthetestisfinished.Thetestmethodstartsexactlyasmanyproducers asconsumersandeachofthemputsortakesthesamenumberofelements,sothetotalnumberofitemsaddedand removedisthesame.
Tests like PutTakeTest tend to be good at finding safety violations. For example, a common error in implementing semaphoreͲcontrolled buffers is to forget that the code actually doing the insertion and extraction requires mutual exclusion(usingsynchronizedorReentrantLock).AsamplerunofPutTakeTestwithaversionofBoundedBufferthat omits making doInsert and doExtract synchronized fails fairly quickly. Running PutTakeTest with a few dozen threadsiteratingafewmilliontimesonbuffersofvariouscapacityonvarioussystemsincreasesourconfidenceabout thelackofdatacorruptioninputandtake.
Tests should be run on multiprocessor systems to increase the diversity of potential interleavings. However, having more than a few CPUs does not necessarily make tests more effective. To maximize the chance of detecting timingͲ
sensitivedataraces,thereshouldbemoreactivethreadsthanCPUs,sothatatanygiventimesomethreadsarerunning andsomeareswitchedout,thusreducingthepredictabilityofinteractionsbetweenthreads.
158 JavaConcurrencyInPractice
Listing12.5.ProducerǦconsumerTestProgramforBoundedBuffer.
public class PutTakeTest {
private static final ExecutorService pool
= Executors.newCachedThreadPool();
private final AtomicInteger putSum = new AtomicInteger(0);
private final AtomicInteger takeSum = new AtomicInteger(0);
private final CyclicBarrier barrier;
private final BoundedBuffer<Integer> bb;
private final int nTrials, nPairs;
public static void main(String[] args) {
new PutTakeTest(10, 10, 100000).test(); // sample parameters
pool.shutdown();
}
PutTakeTest(int capacity, int npairs, int ntrials) {
this.bb = new BoundedBuffer<Integer>(capacity);
this.nTrials = ntrials;
this.nPairs = npairs;
this.barrier = new CyclicBarrier(npairs* 2 + 1);
}
void test() {
try {
for (int i = 0; i < nPairs; i++) {
pool.execute(new Producer());
pool.execute(new Consumer());
}
barrier.await(); // wait for all threads to be ready
barrier.await(); // wait for all threads to finish
assertEquals(putSum.get(), takeSum.get());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
class Producer implements Runnable { /* Listing 12.6 */ }
class Consumer implements Runnable { /* Listing 12.6 */ }
}
Listing12.6.ProducerandConsumerClassesUsedinPutTakeTest.
/* inner classes of PutTakeTest (Listing 12.5) */
class Producer implements Runnable {
public void run() {
try {
int seed = (this.hashCode() ^ (int)System.nanoTime());
int sum = 0;
barrier.await();
for (int i = nTrials; i > 0; --i) {
bb.put(seed);
sum += seed;
seed = xorShift(seed);
}
putSum.getAndAdd(sum);
barrier.await();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
class Consumer implements Runnable {
public void run() {
try {
barrier.await();
int sum = 0;
for (int i = nTrials; i > 0; --i) {
sum += bb.take();
}
takeSum.getAndAdd(sum);
barrier.await();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 159
Inteststhatrununtiltheycompleteafixednumberofoperations,itispossiblethatthetestcasewillneverfinishifthe code being tested encounters an exception due to a bug. The most common way to handle this is to have the test frameworkabortteststhatdonotterminatewithinacertainamountoftime;howlongtowaitshouldbedetermined empirically,andfailuresmustthenbeanalyzedtoensurethattheproblemwasn'tjustthatyoudidn'twaitlongenough.
(Thisproblemisnotuniquetotestingconcurrentclasses;sequentialtestsmustalsodistinguishbetweenlongͲrunning andinfiniteloops.)
12.1.4.TestingResourceManagement
Thetestssofarhavebeenconcernedwithaclass'sadherencetoitsspecificationͲthatitdoeswhatitissupposedtodo.
Asecondaryaspecttotestisthatitdoesnotdothingsitisnotsupposedtodo,suchasleakresources.Anyobjectthat holdsormanagesotherobjectsshouldnotcontinuetomaintainreferencestothoseobjectslongerthannecessary.Such storage leaks prevent garbage collectors from reclaiming memory (or threads, file handles, sockets, database connections,orotherlimitedresources)andcanleadtoresourceexhaustionandapplicationfailure.
ResourcemanagementissuesareespeciallyimportantforclasseslikeBoundedBuffertheentirereasonforboundinga buffer is to prevent application failure due to resource exhaustion when producers get too far ahead of consumers.
Boundingcausesoverlyproductiveproducerstoblockratherthancontinuetocreateworkthatwillconsumemoreand morememoryorotherresources.
UndesirablememoryretentioncanbeeasilytestedwithheapͲinspectiontoolsthatmeasureapplicationmemoryusage; avarietyofcommercialandopenͲsourceheapͲprofilingtoolscandothis.ThetestLeakmethodinListing12.7contains placeholders for a heapͲinspection tool to snapshot the heap, which forces a garbage collection[5] and then records informationabouttheheapsizeandmemoryusage.
[5]Technically,itisimpossibletoforceagarbagecollection;System.gconlysuggeststotheJVMthatthismightbeagoodtimetoperforma garbagecollection.HotSpotcanbeinstructedtoignoreSystem.gccallswith-XX:+DisableExplicitGC.
The testLeak method inserts several large objects into a bounded buffer and then removes them; memory usage at heapsnapshot#2shouldbeapproximatelythesameasatheapsnapshot#1.Ontheotherhand,ifdoExtractforgotto null out the reference to the returned element (items[i]=null), the reported memory usage at the two snapshots woulddefinitelynotbethesame.(Thisisoneofthefewtimeswhereexplicitnullingisnecessary;mostofthetime,itis eithernothelpfuloractuallyharmful[EJItem5].)
12.1.5.UsingCallbacks
CallbackstoclientͲprovidedcodecanbehelpfulinconstructingtestcases;callbacksareoftenmadeatknownpointsin anobject'slifecyclethataregoodopportunitiestoassertinvariants.Forexample,ThreadPoolExecutormakescallsto thetaskRunnablesandtotheThreadFactory.
Listing12.7.TestingforResourceLeaks.
class Big { double[] data = new double[100000]; }
void testLeak() throws InterruptedException {
BoundedBuffer<Big> bb = new BoundedBuffer<Big>(CAPACITY); int heapSize1 = /* snapshot heap */ ;
for (int i = 0; i < CAPACITY; i++)
bb.put(new Big());
for (int i = 0; i < CAPACITY; i++)
bb.take();
int heapSize2 = /* snapshot heap */ ;
assertTrue(Math.abs(heapSize1-heapSize2) < THRESHOLD);
}
Testing a thread pool involves testing a number of elements of execution policy: that additional threads are created when they are supposed to, but not when they are not supposed to; that idle threads get reaped when they are supposedto,etc.Constructingacomprehensivetestsuitethatcoversallthepossibilitiesisamajoreffort,butmanyof themcanbetestedfairlysimplyindividually.
Wecaninstrumentthreadcreationbyusingacustomthreadfactory.TestingThreadFactoryinListing12.8maintainsa count of created threads; test cases can then verify the number of threads created during a test run.
TestingThreadFactorycouldbeextendedtoreturnacustomThreadthatalsorecordswhenthethreadterminates,so thattestcasescanverifythatthreadsarereapedinaccordancewiththeexecutionpolicy.
160 JavaConcurrencyInPractice
Listing12.8.ThreadFactoryforTestingThreadPoolExecutor.
class TestingThreadFactory implements ThreadFactory {
public final AtomicInteger numCreated = new AtomicInteger();
private final ThreadFactory factory
= Executors.defaultThreadFactory();
public Thread newThread(Runnable r) {
numCreated.incrementAndGet();
return factory.newThread(r);
}
}
Ifthecorepoolsizeissmallerthanthemaximumsize,thethreadpoolshouldgrowasdemandforexecutionincreases.
SubmittinglongͲrunningtaskstothepoolmakesthenumberofexecutingtasksstayconstantforlongenoughtomakea fewassertions,suchastestingthatthepoolisexpandedasexpected,asshowninListing12.9.
Listing12.9.TestMethodtoVerifyThreadPoolExpansion.
public void testPoolExpansion() throws InterruptedException {
int MAX_SIZE = 10;
ExecutorService exec = Executors.newFixedThreadPool(MAX_SIZE);
for (int i = 0; i < 10* MAX_SIZE; i++)
exec.execute(new Runnable() {
public void run() {
try {
Thread.sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
for (int i = 0;
i < 20 && threadFactory.numCreated.get() < MAX_SIZE; i++)
Thread.sleep(100);
assertEquals(threadFactory.numCreated.get(), MAX_SIZE);
exec.shutdownNow();
}
12.1.6.GeneratingMoreInterleavings
Sincemanyofthe potentialfailuresin concurrent codearelowͲprobabilityevents,testingforconcurrencyerrorsisa numbersgame,buttherearesomethingsyoucandotoimproveyourchances.We'vealreadymentionedhowrunning on multiprocessor systems with fewer processors than active threads can generate more interleavings than either a singleͲprocessorsystemoronewithmanyprocessors.Similarly,testingonavarietyofsystemswithdifferentprocessor counts,operatingsystems,andprocessorarchitecturescandiscloseproblemsthatmightnotoccuronallsystems.
A useful trick for increasing the number of interleavings, and therefore more effectively exploring the state space of yourprograms,istouseThread.yieldtoencouragemorecontextswitchesduringoperationsthataccesssharedstate.
(The effectiveness of this technique is platformͲspecific, since the JVM is free to treat Thread.yield as a noͲop [JLS
17.9];usingashortbutnonzerosleepwouldbeslowerbutmorereliable.)ThemethodinListing12.10transferscredits fromoneaccounttoanother;betweenthetwoupdateoperations,invariantssuchas"sumofallaccountsequalszero"
donothold.Bysometimesyieldinginthemiddleofanoperation,youmayactivatetimingͲsensitivebugsincodethat doesnotuseadequatesynchronizationtoaccessstate.Theinconvenienceofaddingthesecallsfortestingandremoving themforproductioncanbereducedbyaddingthemusingaspectͲorientedprogramming(AOP)tools.
Listing12.10.UsingThread.yieldtoGenerateMoreInterleavings.
public synchronized void transferCredits(Account from,
Account to,
int amount) {
from.setBalance(from.getBalance() - amount);
if (random.nextInt(1000) > THRESHOLD)
Thread.yield();
to.setBalance(to.getBalance() + amount);
}
12.2.TestingforPerformance
Performancetestsareoftenextendedversionsoffunctionalitytests.Infact,itisalmostalwaysworthwhiletoinclude somebasicfunctionalitytestingwithinperformanceteststoensurethatyouarenottestingtheperformanceofbroken code.
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 161
Whilethereisdefinitelyoverlapbetweenperformanceandfunctionalitytests,theyhavedifferentgoals.Performance testsseektomeasureendͲtoͲendperformancemetricsforrepresentativeusecases.Pickingareasonablesetofusage scenarios is not always easy; ideally, tests should reflect how the objects being tested are actually used in your application.
Insomecasesanappropriatetestscenarioisobvious.BoundedbuffersarenearlyalwaysusedinproducerͲconsumer designs, so it is sensible to measure the throughput of producers feeding data to consumers. We can easily extend PutTakeTesttobecomeaperformancetestforthisscenario.
Acommonsecondarygoalofperformancetestingistoselectsizingsempiricallyforvariousboundsnumbersofthreads, buffercapacities,andsoon.Whilethesevaluesmightturnouttobesensitiveenoughtoplatformcharacteristics(such asprocessortypeorevenprocessorsteppinglevel,numberofCPUs,ormemorysize)torequiredynamicconfiguration, itisequallycommonthatreasonablechoicesforthesevaluesworkwellacrossawiderangeofsystems.
12.2.1.ExtendingPutTakeTesttoAddTiming
TheprimaryextensionwehavetomaketoPutTakeTestistomeasurethetimetakenforarun.Ratherthanattempting tomeasurethetimeforasingleoperation,wegetamoreaccuratemeasurebytimingtheentirerunanddividingbythe numberofoperationstogetaperͲoperationtime.WearealreadyusingaCyclicBarriertostartandstoptheworker threads,sowecanextendthisbyusingabarrieractionthatmeasuresthestartandendtime,asshowninListing12.11.
WecanmodifytheinitializationofthebarriertousethisbarrieractionbyusingtheconstructorforCyclicBarrierthat acceptsabarrieraction:
Listing12.11.BarrierǦbasedTimer.
this.timer = new BarrierTimer();
this.barrier = new CyclicBarrier(npairs * 2 + 1, timer);
public class BarrierTimer implements Runnable {
private boolean started;
private long startTime, endTime;
public synchronized void run() {
long t = System.nanoTime();
if (!started) {
started = true;
startTime = t;
} else
endTime = t;
}
public synchronized void clear() {
started = false;
}
public synchronized long getTime() {
return endTime - startTime;
}
}
ThemodifiedtestmethodusingthebarrierͲbasedtimerisshowninListing12.12.
WecanlearnseveralthingsfromrunningTimedPutTakeTest.OneisthethroughputoftheproducerͲconsumerhandoff operationforvariouscombinationsofparameters;anotherishowtheboundedbufferscaleswithdifferentnumbersof threads;athirdishowwemightselecttheboundsize.Answeringthesequestionsrequiresrunningthetestforvarious combinationsofparameters,sowe'llneedamaintestdriver,showninListing12.13.
Figure 12.1 shows some sample results on a 4Ͳway machine, using buffer capacities of 1, 10, 100, and 1000. We see immediatelythatabuffersizeofonecausesverypoorthroughput;thisisbecauseeachthreadcanmakeonlyatinybit of progress before blocking and waiting for another thread. Increasing buffer size to ten helps dramatically, but increasespasttenofferdiminishingreturns.
162 JavaConcurrencyInPractice
Figure12.1. TimedPutTakeTestwithVariousBufferCapacities.
Itmaybesomewhatpuzzlingatfirstthataddingalotmorethreadsdegradesperformanceonlyslightly.Thereasonis hardtoseefromthedata,buteasytoseeonaCPUperformancemetersuchasperfbarwhilethetestisrunning:even withmanythreads,notmuchcomputationisgoingon,andmostofitisspentblockingandunblockingthreads.Sothere isplentyofCPUslackformorethreadstodothesamethingwithouthurtingperformanceverymuch.
However, be careful about concluding from this data that you can always add more threads to a producerͲconsumer program that uses a bounded buffer. This test is fairly artificial in how it simulates the application; the producers do almost no work to generate the item placed on the queue, and the consumers do almost no work with the item retrieved. If the worker threads in a real producerͲconsumer application do some nontrivial work to produce and consumeitems(asisgenerallythecase),thenthisslackwoulddisappearandtheeffectsofhavingtoomanythreads could be very noticeable. The primary purpose of this test is to measure what constraints the producerͲconsumer handoffviatheboundedbufferimposesonoverallthroughput.
Listing12.12.TestingwithaBarrierǦbasedTimer.
public void test() {
try {
timer.clear();
for (int i = 0; i < nPairs; i++) {
pool.execute(new Producer());
pool.execute(new Consumer());
}
barrier.await();
barrier.await();
long nsPerItem = timer.getTime() / (nPairs* (long)nTrials);
System.out.print("Throughput: " + nsPerItem + " ns/item"); assertEquals(putSum.get(), takeSum.get());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 163
Listing12.13.DriverProgramǦforTimedPutTakeTest.
public static void main(String[] args) throws Exception {
int tpt = 100000; // trials per thread
for (int cap = 1; cap <= 1000; cap*= 10) {
System.out.println("Capacity: " + cap);
for (int pairs = 1; pairs <= 128; pairs*= 2) {
TimedPutTakeTest t =
new TimedPutTakeTest(cap, pairs, tpt);
System.out.print("Pairs: " + pairs + "\t");
t.test();
System.out.print("\t");
Thread.sleep(1000);
t.test();
System.out.println();
Thread.sleep(1000);
}
}
pool.shutdown();
}
12.2.2.ComparingMultipleAlgorithms
While BoundedBuffer is a fairly solid implementation that performs reasonably well, it turns out to be no match for either ArrayBlockingQueue or LinkedBlockingQueue (which explains why this buffer algorithm wasn't selected for inclusionintheclasslibrary).Thejava.util.concurrentalgorithmshavebeenselectedandtuned,inpartusingtests just like those described here, to be as efficient as we know how to make them, while still offering a wide range of functionality.[6]ThemainreasonBoundedBufferfarespoorlyisthatputandtakeeachhavemultipleoperationsthat could encounter contention Ͳ acquire a semaphore, acquire a lock, release a semaphore. Other implementation approacheshavefewerpointsatwhichtheymightcontendwithanotherthread.
[6]Youmightbeabletooutperformthemifyoubothareaconcurrencyexpertandcangiveupsomeoftheprovidedfunctionality.
Figure 12.2 shows comparative throughput on a dual hyperͲthreaded machine for all three classes with 256Ͳelement buffers, using a variant of TimedPutTakeTest. This test suggests that LinkedBlockingQueue scales better than ArrayBlockingQueue. This may seem odd at first: a linked queue must allocate a link node object for each insertion, andhenceseemstobedoingmoreworkthanthearrayͲbasedqueue.However,eventhoughithasmoreallocationand GCoverhead,alinkedqueueallowsmoreconcurrentaccessbyputsandtakesthananarrayͲbasedqueuebecausethe best linked queue algorithms allow the head and tail to be updated independently. Because allocation is usually threadlocal, algorithms that can reduce contention by doing more allocation usually scale better. (This is another instanceinwhichintuitionbasedontraditionalperformancetuningrunscountertowhatisneededforscalability.) Figure12.2.ComparingBlockingQueueImplementations.
12.2.3.MeasuringResponsiveness
So far we have focused on measuring throughput, which is usually the most important performance metric for concurrent programs. But sometimes it is more important to know how long an individual action might take to complete,andinthiscasewewanttomeasurethevarianceofservicetime.Sometimesitmakessensetoallowalonger average service time if it lets us obtain a smaller variance; predictability is avaluable performance characteristic too.
Measuring variance allows us to estimate the answers to qualityͲofͲservice questions like "What percentage of operationswillsucceedinunder100milliseconds?"
164 JavaConcurrencyInPractice
Histogramsoftaskcompletiontimesarenormallythebestwaytovisualizevarianceinservicetime.Variancesareonly slightly more difficult to measure than averages Ͳyou need to keep track of perͲtask completion times in addition to aggregatecompletiontime.Sincetimergranularitycanbeafactorinmeasuringindividualtasktime(anindividualtask maytakelessthanorclosetothesmallest"timertick",whichwoulddistortmeasurementsoftaskduration),toavoid measurementartifactswecanmeasuretheruntimeofsmallbatchesofputandtakeoperationsinstead.
Figure12.3showstheperͲtaskcompletiontimesofavariantofTimedPutTakeTestusingabuffersizeof1000inwhich each of 256 concurrent tasks iterates only 1000 items for nonͲfair (shaded bars) and fair semaphores (open bars).
(Section13.3explainsfairversusnonͲfairqueuingforlocksandsemaphores.)CompletiontimesfornonͲfairsemaphores range from 104 to 8,714 ms, a factor of over eighty. It is possible to reduce this range by forcing more fairness in concurrency control; this is easy to do in BoundedBuffer by initializing the semaphores to fair mode. As Figure 12.3
shows,thissucceedsingreatlyreducingthevariance(nowrangingonlyfrom38,194to38,207ms),butunfortunately also greatly reduces the throughput. (A longerͲrunning test with more typical kinds of tasks would probably show an evenlargerthroughputreduction.)
Figure12.3.CompletionTimeHistogramforTimedPutTakeTestwithDefault(NonǦfair)andFairSemaphores.
Wesawbeforethatverysmallbuffersizescauseheavycontextswitchingandpoorthroughputeveninnonfairmode, becausenearlyeveryoperationinvolvesacontextswitch.Asanindicationthatthecostoffairnessresultsprimarilyfrom blocking threads, we can rerun this test with a buffer size of one and see that nonfair semaphores now perform comparablytofairsemaphores.Figure12.4showsthatfairnessdoesn'tmaketheaveragemuchworseorthevariance muchbetterinthiscase.
Figure12.4.CompletionTimeHistogramforTimedPutTakeTestwithSingleǦitemBuffers.
So,unlessthreadsarecontinuallyblockinganywaybecauseoftightsynchronizationrequirements,nonfairsemaphores providemuchbetterthroughputandfairsemaphoresprovideslowervariance.Becausetheresultsaresodramatically different,Semaphoreforcesitsclientstodecidewhichofthetwofactorstooptimizefor.
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 165
12.3.AvoidingPerformanceTestingPitfalls
Intheory,developingperformancetestsiseasyfindatypicalusagescenario,writeaprogramthatexecutesthatscenario manytimes,andtimeit.Inpractice,youhavetowatchoutforanumberofcodingpitfallsthatpreventperformance testsfromyieldingmeaningfulresults.
12.3.1.GarbageCollection
Thetimingofgarbagecollectionisunpredictable,sothereisalwaysthepossibilitythatthegarbagecollectorwillrun duringameasuredtestrun.IfatestprogramperformsNiterationsandtriggersnogarbagecollectionbutiterationN+
1wouldtriggeragarbagecollection,asmallvariationinthesizeoftheruncouldhaveabig(butspurious)effectonthe measuredtimeperiteration.
There are two strategies for preventing garbage collection from biasing your results. One is to ensure that garbage collectiondoesnotrunatallduringyourtest(youcaninvoketheJVMwith-verbose:gctofindout);alternatively,you canmakesurethatthegarbagecollectorrunsanumberoftimesduringyourrunsothatthetestprogramadequately reflectsthecostofongoingallocationandgarbagecollection.Thelatterstrategyisoftenbetteritrequiresalongertest andismorelikelytoreflectrealͲworldperformance.
MostproducerͲconsumerapplicationsinvolveafairamountofallocationandgarbagecollectionproducersallocatenew objectsthatareusedanddiscardedbyconsumers.Runningtheboundedbuffertestforlongenoughtoincurmultiple garbagecollectionsyieldsmoreaccurateresults.
12.3.2.DynamicCompilation
WritingandinterpretingperformancebenchmarksfordynamicallycompiledlanguageslikeJavaisfarmoredifficultthan for statically compiled languages like C or C++. The HotSpot JVM (and other modern JVMs) uses a combination of bytecodeinterpretationanddynamiccompilation.Whenaclassisfirstloaded,theJVMexecutesitbyinterpretingthe bytecode. At some point, if a method is run often enough, the dynamic compiler kicks in and converts it to machine code;whencompilationcompletes,itswitchesfrominterpretationtodirectexecution.
Thetimingofcompilationisunpredictable.Yourtimingtestsshouldrunonlyafterallcodehasbeencompiled;thereis no value in measuring the speed of the interpreted code since most programs run long enough that all frequently executedcodepathsarecompiled.Allowingthecompilertorunduringameasuredtestruncanbiastestresultsintwo ways:compilationconsumesCPUresources,andmeasuringtheruntimeofacombinationofinterpretedandcompiled code is not a meaningful performance metric. Figure 12.5 shows how this can bias your results. The three timelines representtheexecutionofthesamenumberofiterations:timelineArepresentsallinterpretedexecution,Brepresents compilationhalfwaythroughtherun,andCrepresentscompilationearlyintherun.Thepointatwhichcompilationruns seriouslyaffectsthemeasuredperͲoperationruntime.[7]
[7]TheJVMmaychoosetoperformcompilationintheapplicationthreadorinthebackgroundthread;eachcanbiastimingresultsindifferent ways.
Figure12.5.ResultsBiasedbyDynamicCompilation.
Codemayalsobedecompiled(revertingtointerpretedexecution)andrecompiledforvariousreasons,suchasloadinga class that invalidates assumptions made by prior compilations, or gathering sufficient profiling data to decide that a codepathshouldberecompiledwithdifferentoptimizations.
One way to prevent compilation from biasing your results is to run your program for a long time (at least several minutes) so that compilation and interpreted execution represent a small fraction of the total run time. Another approachistouseanunmeasured"warmͲup"run,inwhichyourcodeisexecutedenoughtobefullycompiledwhen
166 JavaConcurrencyInPractice
youactuallystarttiming.OnHotSpot,runningyourprogramwith-XX:+PrintCompilationprintsoutamessagewhen dynamiccompilationruns,soyoucanverifythatthisispriorto,ratherthanduring,measuredtestruns.
RunningthesametestseveraltimesinthesameJVMinstancecanbeusedtovalidatethetestingmethodology.Thefirst groupofresultsshouldbediscardedaswarmͲup;seeinginconsistentresultsintheremaininggroupssuggeststhatthe testshouldbeexaminedfurthertodeterminewhythetimingresultsarenotrepeatable.
TheJVMusesvariousbackgroundthreadsforhousekeepingtasks.Whenmeasuringmultipleunrelatedcomputationally intensiveactivitiesinasinglerun,itisagoodideatoplaceexplicitpausesbetweenthemeasuredtrialstogivetheJVM
achancetocatchupwithbackgroundtaskswithminimalinterferencefrommeasuredtasks.(Whenmeasuringmultiple relatedactivities,however,suchasmultiplerunsofthesametest,excludingJVMbackgroundtasksinthiswaymaygive unrealisticallyoptimisticresults.)
12.3.3.UnrealisticSamplingofCodePaths
Runtime compilers use profiling information to help optimize the code being compiled. The JVM is permitted to use informationspecifictotheexecutioninordertoproduce bettercode,which meansthat compiling methodMinone program may generate different code than compiling M in another. In some cases, the JVM may make optimizations basedonassumptionsthatmayonlybetruetemporarily,andlaterbackthemoutbyinvalidatingthecompiledcodeif theybecomeuntrue.[8]
[8]Forexample,theJVMcanusemonomorphiccalltransformationtoconvertavirtualmethodcalltoadirectmethodcallifnoclassescurrently loadedoverridethatmethod,butitinvalidatesthecompiledcodeifaclassissubsequentlyloadedthatoverridesthemethod.
As a result, it is important that your test programs not only adequately approximate the usage patterns of a typical application, but also approximate the set of code paths used by such an application. Otherwise, a dynamic compiler couldmakespecialoptimizationstoapurelysingleͲthreadedtestprogramthatcouldnotbeappliedinrealapplications containingatleastoccasionalparallelism.Therefore,testsofmultithreadedperformanceshouldnormallybemixedwith testsofsingleͲthreadedperformance,evenifyouwanttomeasureonlysingleͲthreadedperformance.(Thisissuedoes notariseinTimedPutTakeTestbecauseeventhesmallesttestcaseusestwothreads.) 12.3.4.UnrealisticDegreesofContention
Concurrentapplicationstendtointerleavetwoverydifferentsortsofwork:accessingshareddata,suchasfetchingthe nexttaskfromasharedworkqueue,andthreadͲlocalcomputation(executingthetask,assumingthetaskitselfdoesnot access shared data). Depending on the relative proportions of the two types of work, the application will experience differentlevelsofcontentionandexhibitdifferentperformanceandscalingbehaviors.
IfNthreadsarefetchingtasksfromasharedworkqueueandexecutingthem,andthetasksarecomputeͲintensiveand longͲrunning(anddonotaccessshareddataverymuch),therewillbealmostnocontention;throughputisdominated bytheavailabilityofCPUresources.Ontheotherhand,ifthetasksareveryshortͲlived,therewillbealotofcontention fortheworkqueueandthroughputisdominatedbythecostofsynchronization.
Toobtainrealisticresults,concurrentperformancetestsshouldtrytoapproximatethethreadͲlocalcomputationdone byatypicalapplicationinadditiontotheconcurrentcoordinationunderstudy.Ifthetheworkdoneforeachtaskinan application is significantly different in nature or scope from the test program, it is easy to arrive at unwarranted conclusionsaboutwheretheperformancebottleneckslie.WesawinSection11.5that,forlockͲbasedclassessuchas the synchronized Map implementations, whether access to the lock is mostly contended or mostly uncontended can have a dramatic effect on throughput. The tests in that section do nothing but pound on the Map; even with two threads,allattempts toaccessthe Maparecontended. However,ifanapplicationdidasignificantamountofthreadͲ
localcomputationforeachtimeitaccessestheshareddatastructure,thecontentionlevelmightbelowenoughtooffer goodperformance.
Inthisregard,TimedPutTakeTestmaybeapoormodelforsomeapplications.Sincetheworkerthreadsdonotdovery much, throughput is dominated by coordination overhead, and this is not necessarily the case in all applications that exchangedatabetweenproducersandconsumersviaboundedbuffers.
12.3.5.DeadCodeElimination
Oneofthechallengesofwritinggoodbenchmarks(inanylanguage)isthatoptimizingcompilersareadeptatspotting andeliminatingdeadcodeͲcodethathasnoeffectontheoutcome.Sincebenchmarksoftendon'tcomputeanything, theyareaneasytargetfortheoptimizer.Mostofthetime,itisagoodthingwhentheoptimizerprunesdeadcodefrom aprogram,butforabenchmarkthisisabigproblembecausethenyouaremeasuringlessexecutionthanyouthink.If you'relucky,theoptimizerwillpruneawayyourentireprogram,andthenitwillbeobviousthatyourdataisbogus.If
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 167
you'reunlucky,deadͲcodeeliminationwilljustspeedupyourprogrambysomefactorthatcouldbeexplainedbyother means.
DeadͲcodeeliminationisaprobleminbenchmarkingstaticallycompiledlanguagestoo,butdetectingthatthecompiler haseliminatedagoodchunkofyourbenchmarkisaloteasierbecauseyoucanlookatthemachinecodeandseethata partofyourprogramismissing.Withdynamicallycompiledlanguages,thatinformationisnoteasilyaccessible.
ManymicroͲbenchmarksperformmuch"better"whenrunwithHotSpot's-servercompilerthanwith-client,notjust because the server compiler can produce more efficient code, but also because it is more adept at optimizing dead code.Unfortunately,thedeadͲcodeeliminationthatmadesuchshortworkofyourbenchmarkwon'tdoquiteaswell withcodethatactuallydoessomething.Butyoushouldstillprefer-serverto-clientforbothproductionandtesting onmultiprocessorsystemsͲyoujusthavetowriteyourtestssothattheyarenotsusceptibletodeadͲcodeelimination.
Writingeffectiveperformancetestsrequirestrickingtheoptimizerintonotoptimizingawayyourbenchmarkasdead code. This requires every computed result to be used somehow by your program Ͳ in a way that does not require synchronizationorsubstantialcomputation.
In PutTakeTest, we compute the checksum of elements added to and removed from the queue and combine these checksumsacrossallthethreads,butthiscouldstillbeoptimizedawayifwedonotactuallyusethechecksumvalue.
Wehappentoneedittoverifythecorrectnessofthealgorithm,butyoucanensurethatavalueisusedbyprintingit out. However, you should avoid doing I/O while the test is actually running, so as not to distort the run time measurement.
A cheap trick for preventing a calculation from being optimized away without introducing too much overhead is to computethehashCodeofthefieldofsomederivedobject,compareittoanarbitraryvaluesuchasthecurrentvalueof System. nanoTime,andprintauselessandignorablemessageiftheyhappentomatch: if (foo.x.hashCode() == System.nanoTime())
System.out.print(" ");
Thecomparisonwillrarelysucceed,andifitdoes,itsonlyeffectwillbetoinsertaharmlessspacecharacterintothe output. (The print method buffers output until println is called, so in the rare case that hashCode and System.nanoTimeareequalnoI/Oisactuallyperformed.)
Not only should every computed result be used, but results should also be unguessable. Otherwise, a smart dynamic optimizingcompilerisallowedtoreplaceactionswithpreͲcomputedresults.Weaddressedthisintheconstructionof PutTakeTest,butanytestprogramwhoseinputisstaticdataisvulnerabletothisoptimization.
12.4.ComplementaryTestingApproaches
Whilewe'dliketobelievethataneffectivetestingprogramshould"findallthebugs",thisisanunrealisticgoal.NASA devotesmoreofitsengineeringresourcestotesting(itisestimatedtheyemploy20testersforeachdeveloper)thanany commercialentitycouldaffordtoandthecodeproducedisstillnotfreeofdefects.Incomplexprograms,noamountof testingcanfindallcodingerrors.
Thegoaloftestingisnotsomuchtofinderrorsasitistoincreaseconfidencethatthecodeworksasexpected.Sinceit is unrealistic to assume you can find all the bugs, the goal of a quality assurance (QA) plan should be to achieve the greatestpossibleconfidencegiventhetestingresourcesavailable.Morethingscangowronginaconcurrentprogram thaninasequentialone,andthereforemoretestingisrequiredtoachievethesamelevelofconfidence.Sofarwe've focusedprimarilyontechniquesforconstructingeffectiveunitandperformancetests.Testingiscriticallyimportantfor building confidence that concurrent classes behave correctly, but should be only one of the QA metholologies you employ.
DifferentQAmethodologiesaremoreeffectiveatfindingsometypesofdefectsandlesseffectiveatfindingothers.By employing complementary testing methodologies such as code review and static analysis, you can achieve greater confidencethanyoucouldwithanysingleapproach.
12.4.1.CodeReview
Aseffectiveandimportantasunitandstresstestsareforfindingconcurrencybugs,theyarenosubstituteforrigorous codereviewbymultiplepeople.(Ontheotherhand,codereviewisnosubstitutefortestingeither.)Youcanandshould designteststomaximizetheirchancesofdiscoveringsafetyerrors,andyoushouldrunthemfrequently,butyoushould notneglecttohaveconcurrentcodereviewedcarefullybysomeonebesidesitsauthor.Evenconcurrencyexpertsmake
168 JavaConcurrencyInPractice
mistakes; taking the time to have someone else review the code is almost always worthwhile. Expert concurrent programmers are better at finding subtle races than are most test programs. (Also, platform issues such as JVM
implementation details or processor memory models can prevent bugs from showing up on particular hardware or software configurations.) Code review also has other benefits; not only can it find errors, but it often improves the qualityofcommentsdescribingtheimplementationdetails,thusreducingfuturemaintenancecostandrisk.
12.4.2.StaticAnalysisTools
As of this writing, static analysis tools are rapidly emerging as an effective complement to formal testing and code review.Staticcodeanalysisistheprocessofanalyzingcodewithoutexecutingit,andcodeauditingtoolscananalyze classestolookforinstancesofcommonbugpatterns.StaticanalysistoolssuchastheopenͲsourceFindBugs[9]contain bugͲpatterndetectorsformanycommoncodingerrors,manyofwhichcaneasilybemissedbytestingorcodereview.
[9]http://findbugs.sourceforge.net
Static analysis tools produce a list of warnings that must be examined by hand to determine whether they represent actualerrors.Historically,toolslikelintproducedsomanyfalsewarningsastoscaredevelopersaway,buttoolslike FindBugs have been tuned to produce many fewer false alarms. Static analysis tools are still somewhat primitive (especially in their integration with development tools and lifecycle), but they are already effective enough to be a valuableadditiontothetestingprocess.
Asofthiswriting,FindBugsincludesdetectorsforthefollowingconcurrencyͲrelatedbugpatterns,andmorearebeing addedallthetime:
Inconsistentsynchronization.Manyobjectsfollowthesynchronizationpolicyofguardingallvariableswiththeobject's intrinsic lock. If a field is accessed frequently but not always with the this lock held, this may indicate that the synchronizationpolicyisnotbeingconsistentlyfollowed.
Analysis tools must guess at the synchronization policy because Java classes do not have formal concurrency specifications. In the future, if annotations such as @GuardedBy are standardized, auditing tools could interpret annotationsratherthanhavingtoguessattherelationshipbetweenvariablesandlocks,thusimprovingthequalityof analysis.
Invoking Thread.run. Thread implements Runnable and therefore has a run method. However, it is almost always a mistaketocallThread.rundirectly;usuallytheprogrammermeanttocallThread.start.
Unreleasedlock.Unlikeintrinsiclocks,explicitlocks(seeChapter13)arenotautomaticallyreleasedwhencontrolexits thescopeinwhichtheywereacquired.Thestandardidiomistoreleasethelockfromafinallyblock;otherwisethe lockcanremainunreleasedintheeventofanException.
Emptysynchronizedblock.WhileemptysynchronizedblocksdohavesemanticsundertheJavaMemoryModel,they arefrequentlyusedincorrectly,andthereareusuallybettersolutionstowhateverproblemthedeveloperwastryingto solve.
DoubleͲchecked locking. DoubleͲchecked locking is a broken idiom for reducing synchronization overhead in lazy initialization(seeSection16.2.4)thatinvolvesreadingasharedmutablefieldwithoutappropriatesynchronization.
Startingathreadfromaconstructor.Startingathreadfromaconstructorintroducestheriskofsubclassingproblems, andcanallowthethisreferencetoescapetheconstructor.
Notificationerrors.ThenotifyandnotifyAllmethodsindicatethatanobject'sstatemayhavechangedinawaythat wouldunblockthreadsthatarewaitingontheassociatedconditionqueue.Thesemethodsshouldbecalledonlywhen thestateassociatedwiththeconditionqueuehaschanged.AsynchronizedblockthatcallsnotifyornotifyAllbut doesnotmodifyanystateislikelytobeanerror.(SeeChapter14.)
Condition wait errors. When waiting on a condition queue, Object.wait or Condition. await should be called in a loop, with the appropriate lock held, after testing some state predicate (see Chapter 14). Calling Object.wait or Condition.await without the lock held, not in a loop, or without testing some state predicate is almost certainly an error.
MisuseofLockandCondition.UsingaLockasthelockargumentforasynchronizedblockislikelytobeatypo,asis callingCondition.waitinsteadofawait(thoughthelatterwouldlikelybecaughtintesting,sinceitwouldthrowan IllegalMonitorStateExceptionthefirsttimeitwascalled).
6BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 169
Sleepingorwaitingwhileholdingalock.CallingThread.sleepwithalockheldcanpreventotherthreadsfrommaking progressforalongtimeandisthereforeapotentiallyseriouslivenesshazard.CallingObject.waitorCondition.await withtwolocksheldposesasimilarhazard.
Spinloops.Codethatdoesnothingbutspin(busywait)checkingafieldforanexpectedvaluecanwasteCPUtimeand,if thefieldisnotvolatile,isnotguaranteedtoterminate.Latchesorconditionwaitsareoftenabettertechniquewhen waitingforastatetransitiontooccur.
12.4.3.AspectǦorientedTestingTechniques
As of this writing, aspectͲoriented programming (AOP) techniques have only limited applicability to concurrency, becausemostpopularAOPtoolsdonotyetsupportpointcutsatsynchronizationpoints.However,AOPcanbeapplied toassertinvariantsorsomeaspectsofcompliancewithsynchronizationpolicies.Forexample,(Laddad,2003)provides an example of using an aspect to wrap all calls to nonͲthreadͲsafe Swing methods with the assertion that the call is occurring in the event thread. As it requires no code changes, this technique is easy to apply and can disclose subtle publicationandthreadͲconfinementerrors.
12.4.4.ProfilersandMonitoringTools
Most commercial profiling tools have some support for threads. They vary in feature set and effectiveness, but can oftenprovideinsightintowhatyourprogramisdoing(althoughprofilingtoolsareusuallyintrusiveandcansubstantially affectprogramtimingandbehavior).Mostofferadisplayshowingatimelineforeachthreadwithdifferentcolorsfor thevariousthreadstates(runnable,blockedwaitingforalock,blockedwaitingforI/O,etc.).Suchadisplaycanshow how effectively your program is utilizing the available CPU resources, and if it is doing badly, where to look for the cause. (Many profilers also claim features for identifying which locks are causing contention, but in practice these featuresareoftenablunterinstrumentthanisdesiredforanalyzingaprogram'slockingbehavior.) ThebuiltͲinJMXagentalsoofferssomelimitedfeaturesformonitoringthreadbehavior.TheThreadInfoclassincludes thethread'scurrentstateand,ifthethreadisblocked,thelockorconditionqueueonwhichitisblocked.Ifthe"thread contentionmonitoring"featureisenabled(itisdisabledbydefaultbecauseofitsperformanceimpact),ThreadInfoalso includesthenumberoftimesthatthethreadhasblockedwaitingforalockornotification,andthecumulativeamount oftimeithasspentwaiting.
Summary
Testingconcurrentprogramsforcorrectnesscanbeextremelychallengingbecausemanyofthepossiblefailuremodes of concurrent programs are lowͲprobability events that are sensitive to timing, load, and other hardͲtoͲreproduce conditions. Further, the testing infrastructure can introduce additional synchronization or timing constraints that can mask concurrency problems in the code being tested. Testing concurrent programs for performance can be equally challenging; Java programs are more difficult to test than programs written in statically compiled languages like C, becausetimingmeasurementscanbeaffectedbydynamiccompilation,garbagecollection,andadaptiveoptimization.
Tohavethebestchanceoffindinglatentbugsbeforetheyoccurinproduction,combinetraditionaltestingtechniques (being careful to avoid the pitfalls discussed here) with code reviews and automated analysis tools. Each of these techniquesfindsproblemsthattheothersarelikelytomiss.
170 JavaConcurrencyInPractice
PartIV:AdvancedTopics
Chapter13.ExplicitLocks
Chapter14.BuildingCustomSynchronizers
Chapter15.AtomicVariablesandNonͲblockingSynchronization
Chapter16.TheJavaMemoryModel
AppendixA.AnnotationsforConcurrency
Bibliography
7BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks 171
Chapter13ǦExplicitLocks
BeforeJava5.0,theonlymechanismsforcoordinatingaccesstoshareddataweresynchronizedandvolatile.Java 5.0addsanotheroption:ReentrantLock.Contrarytowhatsomehavewritten,ReentrantLockisnotareplacementfor intrinsiclocking,butratheranalternativewithadvancedfeaturesforwhenintrinsiclockingprovestoolimited.
13.1.LockandReentrantLock
TheLockinterface,showninListing13.1,definesanumberofabstractlockingoperations.Unlikeintrinsiclocking,Lock offersachoiceofunconditional,polled,timed,andinterruptiblelockacquisition,andalllockandunlockoperationsare explicit. Lock implementations must provide the same memoryͲvisibility semantics as intrinsic locks, but can differ in their locking semantics, scheduling algorithms, ordering guarantees, and performance characteristics.
(Lock.newConditioniscoveredinChapter14.)
Listing13.1. LockInterface.
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long timeout, TimeUnit unit)
throws InterruptedException;
void unlock();
Condition newCondition();
}
ReentrantLock implements Lock, providing the same mutual exclusion and memoryͲvisibility guarantees as synchronized. Acquiring a ReentrantLock has the same memory semantics as entering a synchronized block, and releasing a ReentrantLock has the same memory semantics as exiting a synchronized block. (Memory visibility is coveredinSection3.1andinChapter16.)And,likesynchronized,ReentrantLockoffersreentrantlockingsemantics (seeSection2.3.2).ReentrantLocksupportsallofthelockͲacquisitionmodesdefinedbyLock,providingmoreflexibility fordealingwithlockunavailabilitythandoessynchronized.
Whycreateanewlockingmechanismthatissosimilartointrinsiclocking?Intrinsiclockingworksfineinmostsituations buthassomefunctionallimitationsͲitisnotpossibletointerruptathreadwaitingtoacquirealock,ortoattemptto acquirealockwithoutbeingwillingtowaitforitforever.Intrinsiclocksalsomustbereleasedinthesameblockofcode in which they are acquired; this simplifies coding and interacts nicely with exception handling, but makes nonͲblockͲ
structured locking disciplines impossible. None of these are reasons to abandon synchronized, but in some cases a moreflexiblelockingmechanismoffersbetterlivenessorperformance.
Listing13.2showsthecanonicalformforusingaLock.Thisidiomissomewhatmorecomplicatedthanusingintrinsic locks:thelockmustbereleasedinafinallyblock.Otherwise,thelockwouldneverbereleasediftheguardedcode weretothrowanexception.Whenusinglocking,youmustalsoconsiderwhathappensifanexceptionisthrownoutof the try block; if it is possible for the object to be left in an inconsistent state, additional TRy-catch or TRy-finally blocksmaybeneeded.(Youshouldalwaysconsidertheeffectofexceptionswhenusinganyformoflocking,including intrinsiclocking.)
Failing to usefinally to release a Lock is a ticking time bomb. When it goes off, you will have a hard time tracking downitsoriginastherewillbenorecordofwhereorwhentheLockshouldhavebeenreleased.Thisisonereasonnot to use ReentrantLock as a blanket substitute for synchronized: it is more "dangerous" because it doesn't automaticallycleanupthelockwhencontrolleavestheguardedblock.Whilerememberingtoreleasethelockfroma finallyblockisnotallthatdifficult,itisalsonotimpossibletoforget.[1]
[1]FindBugshasan"unreleasedlock"detectoridentifyingwhenaLockisnotreleasedinallcodepathsoutoftheblockinwhichitwasacquired.
Listing13.2.GuardingObjectStateUsingReentrantLock.
Lock lock = new ReentrantLock();
...
lock.lock();
try {
// update object state
// catch exceptions and restore invariants if necessary
} finally {
lock.unlock();
}
172 JavaConcurrencyInPractice
13.1.1.PolledandTimedLockAcquisition
The timed and polled lockͲacquisition modes provided by TryLock allow more sophisticated error recovery than unconditionalacquisition.Withintrinsiclocks,adeadlockisfatalͲtheonlywaytorecoveristorestarttheapplication, and the only defense is to construct your program so that inconsistent lock ordering is impossible. Timed and polled lockingofferanotheroption:probabilisticdeadlockavoidance.
Using timed or polled lock acquisition (TryLock) lets you regain control if you cannot acquire all the required locks, releasetheonesyoudidacquire,andtryagain(oratleastlogthefailureanddosomethingelse).Listing13.3showsan alternate way of addressing the dynamic ordering deadlock from Section 10.1.2: use TRyLock to attempt to acquire bothlocks,butbackoffandretryiftheycannotbothbeacquired.Thesleeptimehasafixedcomponentandarandom component to reduce the likelihood of livelock. If the locks cannot be acquired within the specified time, transferMoneyreturnsafailurestatussothattheoperationcanfailgracefully.(See[CPJ2.5.1.2]and[CPJ2.5.1.3]for moreexamplesofusingpolledlocksfordeadlockavoidance.)
Timedlocksarealsousefulinimplementingactivitiesthatmanageatimebudget(seeSection6.3.7).Whenanactivity withatimebudgetcallsablockingmethod,itcansupplyatimeoutcorrespondingtotheremainingtimeinthebudget.
Thisletsactivitiesterminateearlyiftheycannotdeliveraresultwithinthedesiredtime.Withintrinsiclocks,thereisno waytocancelalockacquisitiononceitisstarted,sointrinsiclocksputtheabilitytoimplementtimeͲbudgetedactivities atrisk.
ThetravelportalexampleinListing6.17onpage134createsaseparatetaskforeachcarͲrentalcompanyfromwhichit was soliciting bids. Soliciting a bid probably involves some sort of networkͲbased request mechanism, such as a web service request. But soliciting a bid might also require exclusive access to a scarce resource, such as a direct communicationslinetothecompany.
WesawonewaytoensureserializedaccesstoaresourceinSection9.5:asingleͲthreadedexecutor.Anotherapproach istouseanexclusivelocktoguardaccesstotheresource.ThecodeinListing13.4triestosendamessageonashared communicationslineguardedbyaLock,butfailsgracefullyifitcannotdosowithinitstimebudget.ThetimedTRyLock makesitpracticaltoincorporateexclusivelockingintosuchatimeͲlimitedactivity.
13.1.2.InterruptibleLockAcquisition
Just as timed lock acquisition allows exclusive locking to be used within timeͲlimited activities, interruptible lock acquisitionallowslockingtobeusedwithincancellableactivities.Section7.1.6identifiedseveralmechanisms,suchas acquiring an intrinsic lock, that are not responsive to interruption. These nonͲinterruptible blocking mechanisms complicatetheimplementationofcancellabletasks.ThelockInterruptiblymethodallowsyoutotrytoacquirealock while remaining responsive to interruption, and its inclusion in Lock avoids creating another category of nonͲ
interruptibleblockingmechanisms.
7BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks 173
Listing13.3.AvoidingLockǦorderingDeadlockUsingtrylock.
public boolean transferMoney(Account fromAcct,
Account toAcct,
DollarAmount amount,
long timeout,
TimeUnit unit)
throws InsufficientFundsException, InterruptedException {
long fixedDelay = getFixedDelayComponentNanos(timeout, unit);
long randMod = getRandomDelayModulusNanos(timeout, unit);
long stopTime = System.nanoTime() + unit.toNanos(timeout);
while (true) {
if (fromAcct.lock.tryLock()) {
try {
if (toAcct.lock.tryLock()) {
try {
if (fromAcct.getBalance().compareTo(amount)
< 0)
throw new InsufficientFundsException();
else {
fromAcct.debit(amount);
toAcct.credit(amount);
return true;
}
} finally {
toAcct.lock.unlock();
}
}
} finally {
fromAcct.lock.unlock();
}
}
if (System.nanoTime() < stopTime)
return false;
NANOSECONDS.sleep(fixedDelay + rnd.nextLong() % randMod);
}
}
Listing13.4.LockingwithaTimeBudget.
public boolean trySendOnSharedLine(String message,
long timeout, TimeUnit unit)
throws InterruptedException {
long nanosToLock = unit.toNanos(timeout)
- estimatedNanosToSend(message);
if (!lock.tryLock(nanosToLock, NANOSECONDS))
return false;
try {
return sendOnSharedLine(message);
} finally {
lock.unlock();
}
}
Thecanonicalstructureofinterruptiblelockacquisitionisslightlymorecomplicatedthannormallockacquisition,astwo TRy blocks are needed. (If the interruptible lock acquisition can throw InterruptedException, the standard try-finallylockingidiomworks.)Listing13.5useslockInterruptiblytoimplementsendOnSharedLinefromListing13.4
sothatwecancallitfromacancellabletask.ThetimedTRyLockisalsoresponsivetointerruptionandsocanbeused whenyouneedbothtimedandinterruptiblelockacquisition.
Listing13.5.InterruptibleLockAcquisition.
public boolean sendOnSharedLine(String message)
throws InterruptedException {
lock.lockInterruptibly();
try {
return cancellableSendOnSharedLine(message);
} finally {
lock.unlock();
}
}
private boolean cancellableSendOnSharedLine(String message)
throws InterruptedException { ... }
13.1.3.NonǦblockǦstructuredLocking
Withintrinsiclocks,acquireͲreleasepairsareblockͲstructuredalockisalwaysreleasedinthesamebasicblockinwhich it was acquired, regardless of how control exits the block. Automatic lock release simplifies analysis and prevents potentialcodingerrors,butsometimesamoreflexiblelockingdisciplineisneeded.
174 JavaConcurrencyInPractice
InChapter11,wesawhowreducinglockgranularitycanenhancescalability.Lockstripingallowsdifferenthashchainsin ahashͲbasedcollectiontousedifferentlocks.Wecanapplyasimilarprincipletoreducelockinggranularityinalinked list by using a separate lock for each link node, allowing different threads to operate independently on different portions of the list. The lock for a given node guards the link pointers and the data stored in that node, so when traversingormodifyingthelistwemustholdthelockononenodeuntilweacquirethelockonthenextnode;onlythen canwereleasethelockonthefirstnode.Anexampleofthistechnique,calledhandͲoverͲhandlockingorlockcoupling, appearsin[CPJ2.5.1.4].
13.2.PerformanceConsiderations
When ReentrantLock was added in Java 5.0, it offered far better contended performance than intrinsic locking. For synchronization primitives, contended performance is the key to scalability: if more resources are expended on lock managementandscheduling,fewerareavailablefortheapplication.Abetterlockimplementationmakesfewersystem calls, forces fewer context switches, and initiates less memoryͲsynchronization traffic on the shared memory bus, operationsthataretimeͲconsuminganddivertcomputingresourcesfromtheprogram.
Java6usesanimprovedalgorithmformanagingintrinsiclocks,similartothatusedbyReentrantLock,thatclosesthe scalabilitygapconsiderably.Figure13.1showstheperformancedifferencebetweenintrinsiclocksandReentrantLock onJava5.0andonaprereleasebuildofJava6onafourͲwayOpteronsystemrunningSolaris.Thecurvesrepresentthe
"speedup" of ReentrantLock over intrinsic locking on a single JVM version. On Java 5.0, ReentrantLock offers considerably better throughput, but on Java 6, the two are quite close.[2] The test program is the same one used in Section11.5,thistimecomparingthethroughputofaHashMapguardedbyanintrinsiclockandbyaReentrantLock.
[2] Though this particular graph doesn't show it, the scalability difference between Java 5.0 and Java 6 really does come from improvement in intrinsiclocking,ratherthanfromregressioninReentrant-Lock.
Figure13.1.IntrinsicLockingVersusReentrantLockPerformanceonJava5.0andJava6.
OnJava5.0,theperformanceofintrinsiclockingdropsdramaticallyingoingfromonethread(nocontention)tomore thanonethread;theperformanceofReentrantLockdropsfarless,showingitsbetterscalability.ButonJava6,itisa differentstoryintrinsiclocksnolongerfallapartundercontention,andthetwoscalefairlysimilarly.
GraphslikeFigure13.1remindusthatstatementsoftheform"XisfasterthanY"areatbestshortͲlived.Performance andscalabilityaresensitivetoplatformfactorssuchasCPU,processorcount,cachesize,andJVMcharacteristics,allof whichcanchangeovertime.[3]
[3]Whenwestartedthisbook,ReentrantLockseemedthelastwordinlockscalability.Lessthanayearlater,intrinsiclockinggivesitagood runforitsmoney.Performanceisnotjustamovingtarget,itcanbeafastͲmovingtarget.
Performance is a moving target; yesterday's benchmark showing that X is faster than Y may already be out of date today.
7BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks
175
13.3.Fairness
TheReentrantLockconstructoroffersachoiceoftwofairnessoptions:createanonfairlock(thedefault)orafairlock.
Threads acquire a fair lock in the order in which they requested it, whereas a nonfair lock permits barging: threads requesting a lock can jump ahead of the queue of waiting threads if the lock happens to be available when it is requested.(Semaphorealsooffersthechoiceoffairornonfairacquisitionordering.)NonfairReentrantLocksdonotgo out of their way to promote bargingthey simply don't prevent a thread from barging if it shows upat the right time.
With a fair lock, a newly requesting thread is queued if the lock is held by another thread or if threads are queued waitingforthelock;withanonfairlock,thethreadisqueuedonlyifthelockiscurrentlyheld.[4]
[4]ThepolledtryLockalwaysbarges,evenforfairlocks.
Wouldn'twewantalllockstobefair?Afterall,fairnessisgoodandunfairnessisbad,right?(Justaskyourkids.)Whenit comes to locking, though, fairness has a significant performance cost because of the overhead of suspending and resumingthreads.Inpractice,astatisticalfairnessguaranteeͲpromisingthatablockedthreadwilleventuallyacquire thelockͲisoftengoodenough,andisfarlessexpensivetodeliver.Somealgorithmsrelyonfairqueuingtoensuretheir correctness,buttheseareunusual.Inmostcases,theperformancebenefitsofnonͲfairlocksoutweighthebenefitsof fairqueuing.
Figure13.2showsanotherrunoftheMapperformancetest,thistimecomparingHashMapwrappedwithfairandnonͲ
fair ReentrantLocks on a fourͲway Opteron system running Solaris, plotted on a log scale.[5] The fairness penalty is nearlytwoordersofmagnitude.Don'tpayforfairnessifyoudon'tneedit.
[5]ThegraphforConcurrentHashMapisfairlywigglyintheregionbetweenfourandeightthreads.Thesevariationsalmostcertainlycome frommeasurementnoise,whichcouldbeintroducedbycoincidentalinteractionswiththehashcodesoftheelements,threadscheduling,map resizing,garbagecollectionorothermemoryͲsystemeffects,orbytheOSdecidingtorunsomeperiodichousekeepingtaskaroundthetimethat testcaseran.Therealityisthatthereareallsortsofvariationsinperformanceteststhatusuallyaren'tworthbotheringtocontrol.Wemadeno attempttocleanupourgraphsartificially,becauserealͲworldperformancemeasurementsarealsofullofnoise.
Figure13.2.FairVersusNonǦfairLockPerformance.
One reason barging locks perform so much better than fair locks under heavy contention is that there can be a significantdelaybetweenwhenasuspendedthreadisresumedandwhenitactuallyruns.Let'ssaythreadAholdsalock andthreadBasksforthatlock.Sincethelockisbusy,Bissuspended.WhenAreleasesthelock,Bisresumedsoitcan tryagain.Inthemeantime,though,ifthreadCrequeststhelock,thereisagoodchancethatCcanacquirethelock,use it,andreleaseitbeforeBevenfinisheswakingup.Inthiscase,everyonewins:Bgetsthelocknolaterthanitotherwise wouldhave,Cgetsitmuchearlier,andthroughputisimproved.
Fairlockstendtoworkbestwhentheyareheldforarelativelylongtimeorwhenthemeantimebetweenlockrequests isrelativelylong.Inthesecases,theconditionunderwhichbargingprovidesathroughputadvantageͲwhenthelockis unheldbutathreadiscurrentlywakinguptoclaimitͲislesslikelytohold.
LikethedefaultReentrantLock,intrinsiclockingoffersnodeterministicfairnessguarantees,butthestatisticalfairness guaranteesofmostlockingimplementationsaregoodenoughforalmostallsituations.Thelanguagespecificationdoes notrequiretheJVMtoimplementintrinsiclocksfairly,andnoproductionJVMsdo.ReentrantLockdoesnotdepress lockfairnesstonewlowsitonlymakesexplicitsomethingthatwaspresentallalong.
176 JavaConcurrencyInPractice
13.4.ChoosingBetweenSynchronizedandReentrantLock
ReentrantLockprovidesthesamelockingandmemorysemanticsasintrinsiclocking,aswellasadditionalfeaturessuch as timed lock waits, interruptible lock waits, fairness, and the ability to implement nonͲblockͲstructured locking. The performanceofReentrantLockappearstodominatethatofintrinsiclocking,winningslightlyonJava6anddramatically onJava5.0.SowhynotdeprecatesynchronizedandencourageallnewconcurrentcodetouseReentrantLock?Some authorshaveinfactsuggestedthis,treatingsynchronizedasa"legacy"construct.Butthisistakingagoodthingway toofar.
Intrinsic locks still have significant advantages over explicit locks. The notation is familiar and compact, and many existingprogramsalreadyuseintrinsiclockingandmixingthetwocouldbeconfusinganderrorͲprone.Reentrant-Lock isdefinitelyamoredangeroustoolthansynchronization;ifyouforgettowraptheunlockcallinafinallyblock,your code will probably appear to run properly, but you've created a time bomb that may well hurt innocent bystanders.
SaveReentrantLockforsituationsinwhichyouneedsomethingReentrantLockprovidesthatintrinsiclockingdoesn't.
ReentrantLockisanadvancedtoolforsituationswhereintrinsiclockingisnotpractical.Useitifyouneeditsadvanced features:timed,polled,orinterruptiblelockacquisition,fairqueuing,ornonͲblockͲstructuredlocking.Otherwise,prefer synchronized.
Under Java 5.0, intrinsic locking has another advantage over ReentrantLock: thread dumps show which call frames acquiredwhichlocksandcandetectandidentifydeadlockedthreads.TheJVMknowsnothingaboutwhichthreadshold ReentrantLocks and therefore cannot help in debugging threading problems using ReentrantLock. This disparity is addressedinJava6byprovidingamanagementandmonitoringinterfacewithwhichlockscanregister,enablinglocking informationforReentrantLockstoappearinthreaddumpsandthroughothermanagementanddebugginginterfaces.
The availability of this information for debugging is a substantial, if mostly temporary, advantage for synchronized; lockinginformationinthreaddumpshassavedmanyprogrammersfromutterconsternation.ThenonͲblockͲstructured nature of ReentrantLock still means that lock acquisitions cannot be tied to specific stack frames, as they can with intrinsiclocks.
Future performance improvements are likely to favor synchronized over ReentrantLock. Because synchronized is builtintotheJVM,itcanperformoptimizationssuchaslockelisionforthreadͲconfinedlockobjectsandlockcoarsening toeliminatesynchronizationwithintrinsiclocks(seeSection11.3.2);doingthiswithlibraryͲbasedlocksseemsfarless likely. Unless you are deploying on Java 5.0 for the foreseeable future and you have a demonstrated need for ReentrantLock's scalability benefits on that platform, it is not a good idea to choose ReentrantLock over synchronizedforperformancereasons.
13.5.ReadǦwriteLocks
ReentrantLockimplementsastandardmutualͲexclusionlock:atmostonethreadatatimecanholdaReentrantLock.
Butmutualexclusionisfrequentlyastrongerlockingdisciplinethanneededtopreservedataintegrity,andthuslimits concurrencymorethannecessary.Mutualexclusionisaconservativelockingstrategythatpreventswriter/writerand writer/readeroverlap,butalsopreventsreader/readeroverlap.Inmany cases,datastructuresare"readͲmostly"they aremutableandaresometimesmodified,butmostaccessesinvolveonlyreading.Inthesecases,itwouldbeniceto relaxthelockingrequirementstoallowmultiplereaderstoaccessthedatastructureatonce.Aslongaseachthreadis guaranteedanupͲtoͲdateviewofthedataandnootherthreadmodifiesthedatawhilethereadersareviewingit,there will be no problems. This is what readͲwrite locks allow: a resource can be accessed by multiple readers or a single writeratatime,butnotboth.
ReadWriteLock, shown in Listing 13.6, exposes two Lock objectsone for reading and one for writing. To read data guardedbyaReadWriteLockyoumustfirstacquirethereadlock,andtomodifydataguardedbyaReadWriteLockyou must first acquire the write lock. While there may appear to be two separate locks, the read lock and write lock are simplydifferentviewsofanintegratedreadͲwritelockobject.
Listing13.6. ReadWriteLockInterface.
public interface ReadWriteLock {
Lock readLock();
Lock writeLock();
}
7BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks 177
ThelockingstrategyimplementedbyreadͲwritelocksallowsmultiplesimultaneousreadersbutonlyasinglewriter.Like Lock, ReadWriteLock admits multiple implementations that can vary in performance, scheduling guarantees, acquisitionpreference,fairness,orlockingsemantics.
ReadͲwritelocksareaperformanceoptimizationdesignedtoallowgreaterconcurrencyincertainsituations.Inpractice, readͲwrite locks can improve performance for frequently accessed readͲmostly data structures on multiprocessor systems; under other conditions they perform slightly worse than exclusive locks due to their greater complexity.
Whethertheyareanimprovementinanygivensituationisbestdeterminedviaprofiling;becauseReadWriteLockuses Lockforthereadandwriteportionsofthelock,itisrelativelyeasytoswapoutareadͲwritelockforanexclusiveoneif profilingdeterminesthatareadͲwritelockisnotawin.
The interaction between the read and write locks allows for a number of possible implementations. Some of the implementationoptionsforaReadWriteLockare:
Releasepreference.Whenawriterreleasesthewritelockandbothreadersandwritersarequeuedup,whoshouldbe givenpreferenceͲreaders,writers,orwhoeveraskedfirst?
Reader barging. If the lock is held by readers but there are waiting writers, should newly arriving readers be granted immediate access, or should they wait behind the writers? Allowing readers to barge ahead of writers enhances concurrencybutrunstheriskofstarvingwriters.
Reentrancy.Arethereadandwritelocksreentrant?
Downgrading.Ifathreadholdsthewritelock,canitacquirethereadlockwithoutreleasingthewritelock?Thiswould letawriter"downgrade"toareadlockwithoutlettingotherwritersmodifytheguardedresourceinthemeantime.
Upgrading.Canareadlockbeupgradedtoawritelockinpreferencetootherwaitingreadersorwriters?MostreadͲ
write lock implementations do not support upgrading, because without an explicit upgrade operation it is deadlockͲ
prone.(Iftworeaderssimultaneouslyattempttoupgradetoawritelock,neitherwillreleasethereadlock.) ReentrantReadWriteLock provides reentrant locking semantics for both locks. Like ReentrantLock, a ReentrantReadWriteLockcanbeconstructedasnonͲfair(thedefault)orfair.Withafairlock,preferenceisgiventothe threadthathasbeenwaitingthelongest;ifthelockisheldbyreadersandathreadrequeststhewritelock,nomore readersareallowedtoacquirethereadlockuntilthewriterhasbeenservicedandreleasesthewritelock.WithanonͲ
fairlock,theorderinwhichthreadsaregrantedaccessisunspecified.Downgradingfromwritertoreaderispermitted; upgradingfromreadertowriterisnot(attemptingtodosoresultsindeadlock).
LikeReentrantLock,thewritelockinReentrantReadWriteLockhasauniqueownerandcanbereleasedonlybythe threadthatacquiredit.InJava5.0,thereadlockbehavesmorelikeaSemaphorethanalock,maintainingonlythecount ofactivereaders,nottheiridentities.ThisbehaviorwaschangedinJava6tokeeptrackalsoofwhichthreadshavebeen grantedthereadlock.[6]
[6]OnereasonforthischangeisthatunderJava5.0,thelockimplementationcannotdistinguishbetweenathreadrequestingthereadlockforthe firsttimeandareentrantlockrequest,whichwouldmakefairreadͲwritelocksdeadlockͲprone.
ReadͲwritelockscanimproveconcurrencywhenlocksaretypicallyheldforamoderatelylongtimeandmostoperations donotmodifytheguardedresources.ReadWriteMapinListing13.7usesaReentrantReadWriteLocktowrapaMapso that it can be shared safely by multiple readers and still prevent readerͲwriter or writerͲwriter conflicts.[7] In reality, ConcurrentHashMap's performance is so good that you would probably use it rather than this approach if all you neededwasaconcurrenthashͲbasedmap,butthistechniquewouldbeusefulifyouwanttoprovidemoreconcurrent accesstoanalternateMapimplementationsuchasLinkedHashMap.
[7]ReadWriteMapdoesnotimplementMapbecauseimplementingtheviewmethodssuchasentrySetandvalueswouldbedifficultand the"easy"methodsareusuallysufficient.
178 JavaConcurrencyInPractice
Listing13.7.WrappingaMapwithaReadǦwriteLock.
public class ReadWriteMap<K,V> {
private final Map<K,V> map;
private final ReadWriteLock lock = new ReentrantReadWriteLock();
private final Lock r = lock.readLock();
private final Lock w = lock.writeLock();
public ReadWriteMap(Map<K,V> map) {
this.map = map;
}
public V put(K key, V value) {
w.lock();
try {
return map.put(key, value);
} finally {
w.unlock();
}
}
// Do the same for remove(), putAll(), clear()
public V get(Object key) {
r.lock();
try {
return map.get(key);
} finally {
r.unlock();
}
}
// Do the same for other read-only Map methods
}
Figure 13.3 shows a throughput comparison between an ArrayList wrapped with a ReentrantLock and with a ReadWriteLock on a fourͲway Opteron system running Solaris. The test program used here is similar to the Map performancetestwe'vebeenusingthroughoutthebookͲeachoperationrandomlyselectsavalueandsearchesforitin thecollection,andasmallpercentageofoperationsmodifythecontentsofthecollection.
Figure13.3.ReadǦwriteLockPerformance.
Summary
Explicit Locks offer an extended feature set compared to intrinsic locking, including greater flexibility in dealing with lock unavailability and greater control over queuing behavior. But ReentrantLock is not a blanket substitute for synchronized;useitonlywhenyouneedfeaturesthatsynchronizedlacks.
ReadͲwrite locks allow multiple readers to access a guarded object concurrently, offering the potential for improved scalabilitywhenaccessingreadͲmostlydatastructures.
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 179
Chapter14ǦBuildingCustomSynchronizers
The class libraries include a number of stateͲdependent classes Ͳ those having operations with stateͲbased preconditionsͲsuchasFutureTask,Semaphore,andBlockingQueue.Forexample,youcannotremoveanitemfroman emptyqueueorretrievetheresultofataskthathasnotyetfinished;beforetheseoperationscanproceed,youmust waituntilthequeueentersthe"nonempty"stateorthetaskentersthe"completed"state.
The easiest way to construct a stateͲdependent class is usually to build on top of an existing stateͲdependent library class;wedidthisinValueLatchonpage187,usingaCountDownLatchtoprovidetherequiredblockingbehavior.Butif thelibraryclassesdonotprovidethefunctionalityyouneed,youcanalsobuildyourownsynchronizersusingthelowͲ
level mechanisms provided by the language and libraries, including intrinsic condition queues, explicit Condition objects,andtheAbstractQueuedSynchronizerframework.Thischapterexploresthevariousoptionsforimplementing statedependenceandtherulesforusingthestatedependencemechanismsprovidedbytheplatform.
14.1.ManagingStateDependence
InasingleͲthreadedprogram,ifastateͲbasedprecondition(like"theconnectionpoolisnonempty")doesnotholdwhen amethodiscalled,itwillneverbecometrue.Therefore,classesinsequentialprogramscanbecodedtofailwhentheir preconditionsdonothold.Butinaconcurrentprogram,stateͲbasedconditionscanchangethroughtheactionsofother threads: a pool that was empty a few instructions ago can become nonempty because another thread returned an element. StateͲdependent methods on concurrent objects can sometimes get away with failing when their preconditionsarenotmet,butthereisoftenabetteralternative:waitforthepreconditiontobecometrue.
StateͲdependentoperationsthatblockuntiltheoperationcanproceedaremoreconvenientandlesserrorͲpronethan thosethatsimplyfail.ThebuiltͲinconditionqueuemechanismenablesthreadstoblockuntilanobjecthasentereda statethatallowsprogressandtowakeblockedthreadswhentheymaybeabletomakefurtherprogress.Wecoverthe detailsofconditionqueuesinSection14.2,buttomotivatethevalueofanefficientconditionwaitmechanism,wefirst showhowstatedependencemightbe(painfully)tackledusingpollingandsleeping.
AblockingstateͲdependentactiontakestheformshowninListing14.1.Thepatternoflockingissomewhatunusualin that the lock is released and reacquired in the middle of the operation. The state variables that make up the preconditionmustbeguardedbytheobject'slock,sothattheycanremainconstantwhilethepreconditionistested.
Butifthepreconditiondoesnothold,thelockmustbereleasedsoanotherthreadcanmodifytheobjectstateotherwise thepreconditionwillneverbecometrue.Thelockmustthenbereacquiredbeforetestingthepreconditionagain.
Listing14.1.StructureofBlockingStateǦdependentActions.
void blockingAction() throws InterruptedException {
acquire lock on object state
while (precondition does not hold) {
release lock
wait until precondition might hold
optionally fail if interrupted or timeout expires
reacquire lock
}
perform action
}
Bounded buffers such as ArrayBlockingQueue are commonly used in producerͲconsumer designs. A bounded buffer providesputandtakeoperations,eachofwhichhaspreconditions:youcannottakeanelementfromanemptybuffer, nor put an element into a full buffer. State dependent operations can deal with precondition failure by throwing an exceptionorreturninganerrorstatus(makingitthecaller'sproblem),orbyblockinguntiltheobjecttransitionstothe rightstate.
We're going to develop several implementations of a bounded buffer that take different approaches to handling preconditionfailure.EachextendsBaseBoundedBufferinListing14.2,whichimplementsaclassicarrayͲbasedcircular buffer where the buffer state variables (buf, head, tail, and count) are guarded by the buffer's intrinsic lock. It provides synchronized doPut and doTake methods that are used by subclasses to implement the put and take operations;theunderlyingstateishiddenfromthesubclasses.
14.1.1.Example:PropagatingPreconditionFailuretoCallers
GrumpyBoundedBuffer in Listing 14.3 is a crude first attempt at implementing a bounded buffer. The put and take methods are synchronized to ensure exclusive access to the buffer state, since both employ checkͲthenͲact logic in accessingthebuffer.
180 JavaConcurrencyInPractice
Whilethisapproachiseasyenoughtoimplement,itisannoyingtouse.Exceptionsaresupposedtobeforexceptional conditions[EJItem39]."Bufferisfull"isnotanexceptionalconditionforaboundedbufferanymorethan"red"isan exceptionalconditionforatrafficsignal.Thesimplificationinimplementingthebuffer(forcingthecallertomanagethe statedependence)ismorethanmadeupforbythesubstantialcomplicationinusingit,sincenowthecallermustbe preparedtocatchexceptionsandpossiblyretryforeverybufferoperation.[1]AwellͲstructuredcalltotakeisshownin Listing14.4notverypretty,especiallyifputandtakearecalledthroughouttheprogram.
[1]PushingthestatedependencebacktothecalleralsomakesitnearlyimpossibletodothingslikepreserveFIFOordering;byforcingthecallerto retry,youlosetheinformationofwhoarrivedfirst.
Listing14.2.BaseClassforBoundedBufferImplementations.
@ThreadSafe
public abstract class BaseBoundedBuffer<V> {
@GuardedBy("this") private final V[] buf;
@GuardedBy("this") private int tail;
@GuardedBy("this") private int head;
@GuardedBy("this") private int count;
protected BaseBoundedBuffer(int capacity) {
this.buf = (V[]) new Object[capacity];
}
protected synchronized final void doPut(V v) {
buf[tail] = v;
if (++tail == buf.length)
tail = 0;
++count;
}
protected synchronized final V doTake() {
V v = buf[head];
buf[head] = null;
if (++head == buf.length)
head = 0;
--count;
return v;
}
public synchronized final boolean isFull() {
return count == buf.length;
}
public synchronized final boolean isEmpty() {
return count == 0;
}
}
Listing14.3.BoundedBufferthatBalksWhenPreconditionsareNotMet.
@ThreadSafe
public class GrumpyBoundedBuffer<V> extends BaseBoundedBuffer<V> {
public GrumpyBoundedBuffer(int size) { super(size); }
public synchronized void put(V v) throws BufferFullException {
if (isFull())
throw new BufferFullException();
doPut(v);
}
public synchronized V take() throws BufferEmptyException {
if (isEmpty())
throw new BufferEmptyException();
return doTake();
}
}
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 181
Listing14.4.ClientLogicforCallingGrumpyBoundedBuffer.
while (true) {
try {
V item = buffer.take();
// use item
break;
} catch (BufferEmptyException e) {
Thread.sleep(SLEEP_GRANULARITY);
}
}
Avariantofthisapproachistoreturnanerrorvaluewhenthebufferisinthewrongstate.Thisisaminorimprovement inthatitdoesn'tabusetheexceptionmechanismbythrowinganexceptionthatreallymeans"sorry,tryagain",butit doesnotaddressthefundamentalproblem:thatcallersmustdealwithpreconditionfailuresthemselves.[2]
[2]QueueoffersbothoftheseoptionsͲpollreturnsnullifthequeueisempty,andremovethrowsanexceptionͲbutQueueisnotintended for use in producerͲconsumer designs. BlockingQueue, whose operations block until the queue is in the right state to proceed, is a better choicewhenproducersandconsumerswillexecuteconcurrently.
The client code in Listing 14.4 is not the only way to implement the retry logic. The caller could retry the take immediately,withoutsleepingͲanapproachknownasbusywaitingorspinwaiting.Thiscouldconsumequitealotof CPUtimeifthebufferstatedoesnotchangeforawhile.Ontheotherhand,ifthecallerdecidestosleepsoasnotto consumesomuchCPUtime,itcouldeasily"oversleep"ifthebufferstatechangesshortlyafterthecalltosleep.Sothe client code is left with the choice between the poor CPU usage of spinning and the poor responsiveness of sleeping.
(SomewherebetweenbusywaitingandsleepingwouldbecallingThread.yieldineachiteration,whichisahinttothe schedulerthatthiswouldbeareasonabletimetoletanotherthreadrun.Ifyouarewaitingforanotherthreadtodo something,thatsomethingmighthappenfasterifyouyieldtheprocessorratherthanconsumingyourfullscheduling quantum.)
14.1.2.Example:CrudeBlockingbyPollingandSleeping
SleepyBoundedBuffer in Listing 14.5 attempts to spare callers the inconvenience of implementing the retry logic on eachcallbyencapsulatingthesamecrude"pollandsleep"retrymechanismwithintheputandtakeoperations.Ifthe bufferisempty,takesleepsuntilanotherthreadputssomedataintothebuffer;ifthebufferisfull, putsleepsuntil another thread makes room by removing some data. This approach encapsulates precondition management and simplifiesusingthebufferͲdefinitelyastepintherightdirection.
TheimplementationofSleepyBoundedBufferismorecomplicatedthanthepreviousattempt.[3]Thebuffercodemust testtheappropriatestateconditionwiththebufferlockheld,becausethevariablesthatrepresentthestatecondition areguardedbythebufferlock.Ifthetestfails,theexecutingthreadsleepsforawhile,firstreleasingthelocksoother threadscanaccessthebuffer.[4]Oncethethreadwakesup,itreacquiresthelockandtriesagain,alternatingbetween sleepingandtestingthestateconditionuntiltheoperationcanproceed.
[3]WewillspareyouthedetailsofSnowWhite'sotherfiveboundedbufferimplementations,especiallySneezyBoundedBuffer.
[4] It is usually a bad idea for a thread to go to sleep or otherwise block with a lock held, but in this case is even worse because the desired condition(bufferisfull/empty)canneverbecometrueifthelockisnotreleased!
Fromtheperspectiveofthecaller,thisworksnicelyͲiftheoperationcanproceedimmediately,itdoes,andotherwiseit blocksͲandthecallerneednotdealwiththemechanicsoffailureandretry.Choosingthesleepgranularityisatradeoff betweenresponsivenessandCPUusage;thesmallerthesleepgranularity,themoreresponsive,butalsothemoreCPU
resourcesconsumed.Figure14.1showshowsleepgranularitycanaffectresponsiveness:theremaybeadelaybetween whenbufferspacebecomesavailableandwhenthethreadwakesupandchecksagain.
Figure14.1.ThreadOversleepingBecausetheConditionBecameTrueJustAfterItWenttoSleep.
182 JavaConcurrencyInPractice
Listing14.5.BoundedBufferUsingCrudeBlocking.
@ThreadSafe
public class SleepyBoundedBuffer<V> extends BaseBoundedBuffer<V> {
public SleepyBoundedBuffer(int size) { super(size); }
public void put(V v) throws InterruptedException {
while (true) {
synchronized (this) {
if (!isFull()) {
doPut(v);
return;
}
}
Thread.sleep(SLEEP_GRANULARITY);
}
}
public V take() throws InterruptedException {
while (true) {
synchronized (this) {
if (!isEmpty())
return doTake();
}
Thread.sleep(SLEEP_GRANULARITY);
}
}
}
SleepyBoundedBufferalsocreatesanotherrequirementforthecallerͲdealingwithInterruptedException.Whena methodblockswaitingforaconditiontobecometrue,thepolitethingtodoistoprovideacancellationmechanism(see Chapter 7). Like most wellͲbehaved blocking library methods, SleepyBounded-Buffer supports cancellation through interruption,returningearlyandthrowingInterruptedExceptionifinterrupted.
Theseattemptstosynthesizeablockingoperationfrompollingandsleepingwerefairlypainful.Itwouldbenicetohave awayofsuspendingathreadbutensuringthatitisawakenedpromptlywhenacertaincondition(suchasthebuffer beingnolongerfull)becomestrue.Thisisexactlywhatconditionqueuesdo.
14.1.3.ConditionQueuestotheRescue
Conditionqueuesarelikethe"toastisready"bellonyourtoaster.Ifyouarelisteningforit,youarenotifiedpromptly whenyourtoastisreadyandcandropwhatyouaredoing(ornot,maybeyouwanttofinishthenewspaperfirst)and get your toast. If you are not listening for it (perhaps you went outside to get the newspaper), you could miss the notification,butonreturntothekitchenyoucanobservethestateofthetoasterandeitherretrievethetoastifitis finishedorstartlisteningforthebellagainifitisnot.
AconditionqueuegetsitsnamebecauseitgivesagroupofthreadsͲcalledthewaitsetͲawaytowaitforaspecific condition to become true. Unlike typical queues in which the elements are data items, the elements of a condition queuearethethreadswaitingforthecondition.
Just as each Java object can act as a lock, each object can also act as a condition queue, and the wait, notify, and notifyAllmethodsinObjectconstitutetheAPIforintrinsicconditionqueues.Anobject'sintrinsiclockanditsintrinsic conditionqueuearerelated:inordertocallanyoftheconditionqueuemethodsonobjectX,youmustholdthelockon X.ThisisbecausethemechanismforwaitingforstateͲbasedconditionsisnecessarilytightlyboundtothemechanism for preserving state consistency: you cannot wait for a condition unless you can examine the state, and you cannot releaseanotherthreadfromaconditionwaitunlessyoucanmodifythestate.
Object.wait atomically releases the lock and asks the OS to suspend the current thread, allowing other threads to acquirethelockandthereforemodifytheobjectstate.Uponwaking,itreacquiresthelockbeforereturning.Intuitively, calling wait means "I want to go to sleep, but wake me when something interesting happens", and calling the notificationmethodsmeans"somethinginterestinghappened".
BoundedBuffer in Listing 14.6 implements a bounded buffer using wait and notifyAll. This is simpler than the sleepingversion,andisbothmoreefficient(wakinguplessfrequentlyifthe bufferstate doesnotchange)and more responsive(wakinguppromptlywhenaninterestingstatechangehappens).Thisisabigimprovement,butnotethat the introduction of condition queues didn't change the semantics compared to the sleeping version. It is simply an optimization in several dimensions: CPU efficiency, contextͲswitch overhead, and responsiveness. Condition queues
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 183
don'tletyoudoanythingyoucan'tdowithsleepingandpolling[5],buttheymakeitaloteasierandmoreefficientto expressandmanagestatedependence.
[5]Thisisnotquitetrue;afairconditionqueuecanguaranteetherelativeorderinwhichthreadsarereleasedfromthewaitset.Intrinsiccondition queues,likeintrinsiclocks,donotofferfairqueuing;explicitConditionsofferachoiceoffairornonͲfairqueuing.
Listing14.6.BoundedBufferUsingConditionQueues.
@ThreadSafe
public class BoundedBuffer<V> extends BaseBoundedBuffer<V> {
// CONDITION PREDICATE: not-full (!isFull())
// CONDITION PREDICATE: not-empty (!isEmpty())
public BoundedBuffer(int size) { super(size); }
// BLOCKS-UNTIL: not-full
public synchronized void put(V v) throws InterruptedException {
while (isFull())
wait();
doPut(v);
notifyAll();
}
// BLOCKS-UNTIL: not-empty
public synchronized V take() throws InterruptedException {
while (isEmpty())
wait();
V v = doTake();
notifyAll();
return v;
}
}
BoundedBufferisfinallygoodenoughtouseͲitiseasytouseandmanagesstatedependencesensibly.[6]Aproduction version should also include timed versions of put and take, so that blocking operations can time out if they cannot completewithinatimebudget.ThetimedversionofObject.waitmakesthiseasytoimplement.
[6]ConditionBoundedBufferinSection14.3isevenbetter:itismoreefficientbecauseitcanusesinglenotificationinsteadofnotifyAll.
14.2.UsingConditionQueues
ConditionqueuesmakeiteasiertobuildefficientandresponsivestateͲdependentclasses,buttheyarestilleasytouse incorrectly;therearealotofrulesregardingtheirproperusethatarenotenforcedbythecompilerorplatform.(Thisis one of the reasons to build on top of classes like LinkedBlockingQueue, CountDown-Latch, Semaphore, and FutureTaskwhenyoucan;ifyoucangetawaywithit,itisaloteasier.)
14.2.1.TheConditionPredicate
Thekeytousingconditionqueuescorrectlyisidentifyingtheconditionpredicatesthattheobjectmaywaitfor.Itisthe conditionpredicatethatcausesmuchoftheconfusionsurroundingwaitandnotify,becauseithasnoinstantiationin theAPIandnothingineitherthelanguagespecificationortheJVMimplementationensuresitscorrectuse.Infact,itis not mentioned directly at all in the language specification or the Javadoc. But without it, condition waits would not work.
TheconditionpredicateisthepreconditionthatmakesanoperationstateͲdependentinthefirstplace.Inabounded buffer,takecanproceedonlyifthebufferisnotempty;otherwiseitmustwait.Fortake,theconditionpredicateis"the buffer is not empty", which take must test for before proceeding. Similarly, the condition predicate for put is "the buffer is not full". Condition predicates are expressions constructed from the state variables of the class; BaseBoundedBuffer tests for "buffer not empty" by comparing count to zero, and tests for "buffer not full" by comparingcounttothebuffersize.
Documenttheconditionpredicate(s)associatedwithaconditionqueueandtheoperationsthatwaitonthem.
There is an important threeͲway relationship in a condition wait involving locking, the wait method, and a condition predicate. The condition predicate involves state variables, and the state variables are guarded by a lock, so before testingtheconditionpredicate,wemustholdthatlock.Thelockobjectandtheconditionqueueobject(theobjecton whichwaitandnotifyareinvoked)mustalsobethesameobject.
InBoundedBuffer,thebufferstateisguardedbythebufferlockandthebufferobjectisusedastheconditionqueue.
Thetakemethodacquiresthebufferlockandthenteststheconditionpredicate(thatthebufferisnonempty).Ifthe
184 JavaConcurrencyInPractice
buffer is indeed nonempty, it removes the first element, which it can do because it still holds the lock guarding the bufferstate.
Iftheconditionpredicateisnottrue(thebufferisempty),takemustwaituntilanotherthreadputsanobjectinthe buffer. It does this by calling wait on the buffer's intrinsic condition queue, which requires holding the lock on the condition queue object. As careful design would have it, take already holds that lock, which it needed to test the conditionpredicate(andiftheconditionpredicatewastrue,tomodifythebufferstateinthesameatomicoperation).
Thewaitmethodreleasesthelock,blocksthecurrentthread,andwaitsuntilthespecifiedtimeoutexpires,thethread isinterrupted,orthethreadisawakenedbyanotification.Afterthethreadwakesup,waitreacquiresthelockbefore returning.Athreadwakingupfromwaitgetsnospecialpriorityinreacquiringthelock;itcontendsforthelockjustlike anyotherthreadattemptingtoenterasynchronizedblock.
Everycalltowaitisimplicitlyassociatedwithaspecificconditionpredicate.Whencallingwaitregardingaparticular conditionpredicate,thecallermustalreadyholdthelockassociatedwiththeconditionqueue,andthatlockmustalso guardthestatevariablesfromwhichtheconditionpredicateiscomposed.
14.2.2.WakingUpTooSoon
AsifthethreeͲwayrelationshipamongthelock,theconditionpredicate,andtheconditionqueuewerenotcomplicated enough,thatwaitreturnsdoesnotnecessarilymeanthattheconditionpredicatethethreadiswaitingforhasbecome true.
Asingleintrinsicconditionqueuemaybeusedwithmorethanoneconditionpredicate.Whenyourthreadisawakened becausesomeonecallednotifyAll,thatdoesn'tmeanthattheconditionpredicateyouwerewaitingforisnowtrue.
(Thisislikehavingyourtoasterandcoffeemakershareasinglebell;whenitrings,youstillhavetolooktoseewhich deviceraisedthesignal.)[7]Additionally,waitisevenallowedtoreturn"spuriously�”notinresponsetoanythreadcalling notify.[8]
[7]ThissituationactuallydescribesTim'skitchenprettywell;somanydevicesbeepthatwhenyouhearone,youhavetoinspectthetoaster,the microwave,thecoffeemaker,andseveralotherstodeterminethecauseofthesignal.
[8]Topushthebreakfastanalogywaytoofar,thisislikeatoasterwithalooseconnectionthatmakesthebellgooffwhenthetoastisreadybut alsosometimeswhenitisnotready.
When control reͲenters the code calling wait, it has reacquired the lock associated with the condition queue. Is the conditionpredicatenowtrue?Maybe.ItmighthavebeentrueatthetimethenotifyingthreadcallednotifyAll,but could have become false again by the time you reacquire the lock. Other threads may have acquired the lock and changedtheobject'sstatebetweenwhenyourthreadwasawakenedandwhenwaitreacquiredthelock.Ormaybeit hasn'tbeentrueatallsinceyoucalledwait.Youdon'tknowwhyanotherthreadcallednotifyornotifyAll;maybeit was because another condition predicate associated with the same condition queue became true. Multiple condition predicatesperconditionqueuearequitecommonͲBoundedBufferusesthesameconditionqueueforboththe"not full"and"notempty"predicates.[9]
[9]Itisactually possibleforthreadstobewaitingforboth"notfull"and"not empty"atthesametime!Thiscanhappenwhenthenumberof producers/consumersexceedsthebuffercapacity.
Forallthesereasons,whenyouwakeupfromwaityoumusttesttheconditionpredicateagain,andgobacktowaiting (orfail)ifitisnotyettrue.Sinceyoucanwakeuprepeatedly withoutyourconditionpredicatebeingtrue,youmust thereforealwayscallwaitfromwithinaloop,testingtheconditionpredicateineachiteration.Thecanonicalformfora conditionwaitisshowninListing14.7.
Listing14.7.CanonicalFormforStateǦdependentMethods.
void stateDependentMethod() throws InterruptedException {
// condition predicate must be guarded by lock
synchronized(lock) {
while (!conditionPredicate())
lock.wait();
// object is now in desired state
}
}
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 185
Whenusingconditionwaits(Object.waitorCondition.await):
x Alwayshaveaconditionpredicatesometestofobjectstatethatmustholdbeforeproceeding; x Alwaystesttheconditionpredicatebeforecallingwait,andagainafterreturningfromwait; x Alwayscallwaitinaloop;
x Ensure that the state variables making up the condition predicate are guarded by the lock associated with the conditionqueue;
x Holdthelockassociatedwiththetheconditionqueuewhencallingwait,notify,ornotifyAll;and x Donotreleasethelockaftercheckingtheconditionpredicatebutbeforeactingonit.
14.2.3.MissedSignals
Chapter10discussedlivenessfailuressuchasdeadlockandlivelock.Anotherformoflivenessfailureismissedsignals.A missed signal occurs when a thread must wait for a specific condition that is already true, but fails to check the conditionpredicatebeforewaiting.Nowthethreadiswaitingtobenotifiedofaneventthathasalreadyoccurred.This islikestartingthetoast,goingouttogetthenewspaper,havingthebellgooffwhileyouareoutside,andthensitting down at the kitchen table waiting for the toast bell. You could wait a long timepotentially forever.[10] Unlike the marmaladeforyourtoast,notificationisnot"sticky"ifthreadAnotifiesonaconditionqueueandthreadBsubsequently waitsonthatsameconditionqueue,BdoesnotimmediatelywakeupanothernotificationisrequiredtowakeB.Missed signalsaretheresultofcodingerrorslikethosewarnedagainstinthelistabove,suchasfailingtotestthecondition predicatebeforecallingwait.IfyoustructureyourconditionwaitsasinListing14.7,youwillnothaveproblemswith missedsignals.
[10]Inordertoemergefromthiswait,someoneelsewouldhavetomaketoast,butthiswilljustmakemattersworse;whenthebellrings,youwill thenhaveadisagreementabouttoastownership.
14.2.4.Notification
Sofar,we'vedescribed halfofwhat goesoninaconditionwait:waiting. Theotherhalfisnotification.Inabounded buffer,takeblocksifcalledwhenthebufferisempty.Inorderfortaketounblockwhenthebufferbecomesnonempty, we must ensure that every code path in which the buffer could become nonempty performs a notification. In BoundedBuffer,thereisonlyonesuchplaceafteraput.SoputcallsnotifyAllaftersuccessfullyaddinganobjectto thebuffer.Similarly,takecallsnotifyAllafterremovinganelementtoindicatethatthebuffermaynolongerbefull, incaseanythreadsarewaitingonthe"notfull"condition.
Whenever you wait on a condition, make sure that someone will perform a notification whenever the condition predicatebecomestrue.
TherearetwonotificationmethodsintheconditionqueueAPInotifyandnotifyAll.Tocalleither,youmustholdthe lock associated with the condition queue object. Calling notify causes the JVM to select one thread waiting on that conditionqueuetowakeup;callingnotifyAllwakesupallthethreadswaitingonthatconditionqueue.Becauseyou mustholdthelockontheconditionqueueobjectwhencallingnotifyornotifyAll,andwaitingthreadscannotreturn fromwaitwithoutreacquiringthelock,thenotifyingthreadshouldreleasethelockquicklytoensurethatthewaiting threadsareunblockedassoonaspossible.
Becausemultiplethreadscouldbewaitingonthesameconditionqueuefordifferentconditionpredicates,usingnotify instead of notifyAll can be dangerous, primarily because single notification is prone to a problem akin to missed signals.
BoundedBufferprovidesagoodillustrationofwhynotifyAllshouldbepreferredtosinglenotifyinmostcases.The conditionqueueisusedfortwodifferentconditionpredicates:"notfull"and"notempty".SupposethreadAwaitsona conditionqueueforpredicatePA,whilethreadBwaitsonthesameconditionqueueforpredicatePB.Now,supposePB
becomes true and thread C performs a single notify: the JVM will wake up one thread of its own choosing. If A is chosen, it will wake up, see that PA is not yet true, and go back to waiting. Meanwhile, B, which could now make progress, does not wake up. This is not exactly a missed signalit's more of a "hijacked signal"but the problem is the same:athreadiswaitingforasignalthathas(orshouldhave)alreadyoccurred.
SinglenotifycanbeusedinsteadofnotifyAllonlywhenbothofthefollowingconditionshold: x Uniformwaiters.Onlyoneconditionpredicateisassociatedwiththeconditionqueue,andeachthreadexecutesthe samelogicuponreturningfromwait;and
x OneͲin,oneͲout.Anotificationontheconditionvariableenablesatmostonethreadtoproceed.
186 JavaConcurrencyInPractice
BoundedBuffermeetstheoneͲin,oneͲoutrequirement,butdoesnotmeettheuniformwaitersrequirementbecause waiting threads might be waiting for either the "not full" and "not empty" condition. A "starting gate" latch like that usedinTestHarnessonpage96,inwhichasingleeventreleasesasetofthreads,doesnotmeettheoneͲin,oneͲout requirementbecauseopeningthestartinggateletsmultiplethreadsproceed.
Most classes don't meet these requirements, so the prevailing wisdom is to use notifyAll in preference to single notify. While this may be inefficient, it is much easier to ensure that your classes behave correctly when using notifyAllinsteadofnotify.
This "prevailing wisdom" makes some people uncomfortable, and for good reason. Using notifyAll when only one threadcanmakeprogressisinefficientsometimesalittle,sometimesgrosslyso.Iftenthreadsarewaitingonacondition queue,callingnotifyAllcauseseachofthemtowakeupandcontendforthelock;thenmostorallofthemwillgo rightbacktosleep.Thismeansalot ofcontextswitchesand alotofcontendedlockacquisitionsforeachevent that enables(maybe)asinglethreadtomakeprogress.(Intheworstcase,usingnotify-AllresultsinO(n2)wakeupswhere n would suffice.) This is another situation where performance concerns support one approach and safety concerns supporttheother.
The notification done by put and take in BoundedBuffer is conservative: a notification is performed every time an objectisputintoorremovedfromthebuffer.Thiscouldbeoptimizedbyobservingthatathreadcanbereleasedfroma waitonlyifthebuffergoesfromemptytonotemptyorfromfulltonotfull,andnotifyingonlyifaputortakeeffected one of these state transitions. This is called conditional notification. While conditional notification can improve performance, it is tricky to get right (and also complicates the implementation of subclasses) and so should be used carefully.Listing14.8illustratesusingconditionalnotificationinBoundedBuffer.put.
Singlenotificationandconditionalnotificationareoptimizations.Asalways,followtheprinciple"Firstmakeitright,and thenmakeitfastifitisnotalreadyfastenough"whenusingtheseoptimizations;itiseasytointroducestrangeliveness failuresbyapplyingthemincorrectly.
Listing14.8.UsingConditionalNotificationinBoundedBuffer.put.
public synchronized void put(V v) throws InterruptedException {
while (isFull())
wait();
boolean wasEmpty = isEmpty();
doPut(v);
if (wasEmpty)
notifyAll();
}
14.2.5.Example:AGateClass
ThestartinggatelatchinTestHarnessonpage96wasconstructedwithaninitialcountofone,creatingabinarylatch: onewithtwostates,theinitialstateandtheterminalstate.Thelatchpreventsthreadsfrompassingthestartinggate untilitisopened,atwhichpointallthethreadscanpassthrough.Whilethislatchingmechanismisoftenexactlywhatis needed,sometimesitisadrawbackthatagateconstructedinthismannercannotbereclosedonceopened.
ItiseasytodeveloparecloseableThreadGateclassusingconditionwaits,asshowninListing14.9.ThreadGateletsthe gate be opened and closed, providing an await method that blocks until the gate is opened. The open method uses notifyAllbecausethesemanticsofthisclassfailthe"oneͲin,oneͲout"testforsinglenotification.
The condition predicate used by await is more complicated than simply testing isOpen. This is needed because if N
threadsarewaitingatthegateatthetimeitisopened,theyshouldallbeallowedtoproceed.But,ifthegateisopened and closed in rapid succession, all threads might not be released if await examines only isOpen: by the time all the threads receive the notification, reacquire the lock, and emerge from wait, the gate may have closed again. So ThreadGate uses a somewhat more complicated condition predicate: every time the gate is closed, a "generation"
counterisincremented,andathreadmaypassawaitifthegateisopennoworifthegatehasopenedsincethisthread arrivedatthegate.
Since ThreadGate only supports waiting for the gate to open, it performs notification only in open; to support both
"wait for open" and "wait for close" operations, it would have to notify in both open and close. This illustrates why stateͲdependent classes can be fragile to maintainthe addition of a new statedependent operation may require modifyingmanycodepathsthatmodifytheobjectstatesothattheappropriatenotificationscanbeperformed.
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 187
14.2.6.SubclassSafetyIssues
Usingconditionalorsinglenotificationintroducesconstraintsthatcancomplicatesubclassing[CPJ3.3.3.3].Ifyouwant tosupportsubclassingatall,youmuststructureyourclasssosubclassescanaddtheappropriatenotificationonbehalf ofthebaseclassifitissubclassedinawaythatviolatesoneoftherequirementsforsingleorconditionalnotification.
Listing14.9.RecloseableGateUsingWaitandNotifyall.
@ThreadSafe
public class ThreadGate {
// CONDITION-PREDICATE: opened-since(n) (isOpen || generation>n)
@GuardedBy("this") private boolean isOpen;
@GuardedBy("this") private int generation;
public synchronized void close() {
isOpen = false;
}
public synchronized void open() {
++generation;
isOpen = true;
notifyAll();
}
// BLOCKS-UNTIL: opened-since(generation on entry)
public synchronized void await() throws InterruptedException {
int arrivalGeneration = generation;
while (!isOpen && arrivalGeneration == generation)
wait();
}
}
AstateͲdependentclassshouldeitherfullyexpose(anddocument)itswaitingandnotificationprotocolstosubclasses, orpreventsubclassesfromparticipatinginthematall.(Thisisanextensionof"designanddocumentforinheritance,or elseprohibitit"[EJItem15].)Attheveryleast,designingastateͲdependentclassforinheritancerequiresexposingthe conditionqueuesandlocksanddocumentingtheconditionpredicatesandsynchronizationpolicy;itmayalsorequire exposingtheunderlyingstatevariables.(TheworstthingastateͲdependentclasscandoisexposeitsstatetosubclasses but not document its protocols for waiting and notification; this is like a class exposing its state variables but not documentingitsinvariants.)
One option for doing this is to effectively prohibit subclassing, either by making the class final or by hiding the conditionqueues,locks,andstatevariablesfromsubclasses.Otherwise,ifthesubclassdoessomethingtoundermine thewaythebaseclassusesnotify,itneedstobeabletorepairthedamage.Consideranunboundedblockingstackin which the pop operation blocks if the stack is empty but the push operation can always proceed. This meets the requirements for single notification. If this class uses single notification and a subclass adds a blocking "pop two consecutive elements" method, there are now two classes of waiters: those waiting to pop one element and those waiting to pop two. But if the base class exposes the condition queue and documents its protocols for using it, the subclasscanoverridethepushmethodtoperformanotifyAll,restoringsafety.
14.2.7.EncapsulatingConditionQueues
Itisgenerallybesttoencapsulatetheconditionqueuesothatitisnotaccessibleoutsidetheclasshierarchyinwhichitis used.Otherwise,callersmightbetemptedtothinktheyunderstandyourprotocolsforwaitingandnotificationanduse theminamannerinconsistentwithyourdesign.(Itisimpossibletoenforcetheuniformwaitersrequirementforsingle notificationunlesstheconditionqueueobjectisinaccessibletocodeyoudonotcontrol;ifaliencodemistakenlywaits onyourconditionqueue,thiscouldsubvertyournotificationprotocolandcauseahijackedsignal.) Unfortunately,thisadviceͲtoencapsulateobjectsusedasconditionqueuesͲisnotconsistentwiththemostcommon design pattern for threadͲsafe classes, in which an object's intrinsic lock is used to guard its state. BoundedBuffer illustratesthiscommonidiom,wherethebufferobjectitselfisthelockandconditionqueue.However,BoundedBuffer couldbeeasilyrestructuredtouseaprivatelockobjectandconditionqueue;theonlydifferencewouldbethatitwould nolongersupportanyformofclientͲsidelocking.
14.2.8.EntryandExitProtocols
Wellings(Wellings,2004)characterizestheproperuseofwaitandnotifyintermsofentryandexitprotocols.Foreach stateͲdependent operation and for each operation that modifies state on which another operation has a state dependency, you should define and document an entry and exit protocol. The entry protocol is the operation's conditionpredicate;theexitprotocolinvolvesexamininganystatevariablesthathavebeenchangedbytheoperation
188 JavaConcurrencyInPractice
toseeiftheymighthavecausedsomeotherconditionpredicatetobecometrue,andifso,notifyingontheassociated conditionqueue.
AbstractQueuedSynchronizer, upon which most of the stateͲdependent classes in java.util.concurrent are built (see Section 14.4), exploits the concept of exit protocol. Rather than letting synchronizer classes perform their own notification, it instead requires synchronizer methods to return a value indicating whether its action might have unblocked one or more waiting threads. This explicit API requirement makes it harder to "forget" to notify on some statetransitions.
14.3.ExplicitConditionObjects
AswesawinChapter13,explicitLockscanbeusefulinsomesituationswhereintrinsiclocksaretooinflexible.Justas Lockisageneralizationofintrinsiclocks,Condition(seeListing14.10)isageneralizationofintrinsicconditionqueues.
Intrinsic condition queues have several drawbacks. Each intrinsic lock can have only one associated condition queue, whichmeansthatinclasseslikeBoundedBuffermultiplethreadsmightwaitonthesameconditionqueuefordifferent conditionpredicates,andthemostcommonpatternforlockinginvolvesexposingtheconditionqueueobject.Bothof thesefactorsmakeitimpossibletoenforcetheuniformwaiterrequirementforusingnotifyAll.Ifyouwanttowritea concurrent object with multiple condition predicates, or you want to exercise more control over the visibility of the condition queue, the explicit Lock and Condition classes offer a more flexible alternative to intrinsic locks and conditionqueues.
AConditionisassociatedwithasingleLock,justasaconditionqueueisassociatedwithasingleintrinsiclock;tocreate aCondition,callLock.newConditionontheassociatedlock.AndjustasLockoffersaricherfeaturesetthanintrinsic locking,Conditionoffersaricherfeaturesetthanintrinsicconditionqueues:multiplewaitsetsperlock,interruptible anduninterruptibleconditionwaits,deadlineͲbasedwaiting,andachoiceoffairornonfairqueueing.
Listing14.10. ConditionInterface.
public interface Condition {
void await() throws InterruptedException;
boolean await(long time, TimeUnit unit)
throws InterruptedException;
long awaitNanos(long nanosTimeout) throws InterruptedException;
void awaitUninterruptibly();
boolean awaitUntil(Date deadline) throws InterruptedException;
void signal();
void signalAll();
}
Unlikeintrinsicconditionqueues,youcanhaveasmanyConditionobjectsperLockasyouwant.Conditionobjects inheritthefairnesssettingoftheirassociatedLock;forfairlocks,threadsarereleasedfromCondition.awaitinFIFO
order.
Hazard warning: The equivalents of wait, notify, and notifyAll for Condition objects are await, signal, and signalAll.However,ConditionextendsObject,whichmeansthatitalsohaswaitandnotifymethods.Besureto usetheproperversionsͲawaitandsignal - instead!
Listing 14.11 shows yet another bounded buffer implementation, this time using two Conditions, notFull and notEmpty, to represent explicitly the "not full" and "not empty" condition predicates. When take blocks because the bufferisempty,itwaitsonnotEmpty,andputunblocksanythreadsblockedintakebysignalingonnotEmpty.
The behavior of ConditionBoundedBuffer is the same as BoundedBuffer, but its use of condition queues is more readable Ͳ it is easier to analyze a class that uses multiple Conditions than one that uses a single intrinsic condition queue with multiple condition predicates. By separating the two condition predicates into separate wait sets, Conditionmakesiteasiertomeettherequirementsforsinglenotification.Usingthemoreefficientsignalinsteadof signalAllreducesthenumberofcontextswitchesandlockacquisitionstriggeredbyeachbufferoperation.
JustaswithbuiltͲinlocksandconditionqueues,thethreeͲwayrelationshipamongthelock,theconditionpredicate,and theconditionvariablemustalsoholdwhenusingexplicitLocksandConditions.Thevariablesinvolvedinthecondition predicate must be guarded by the Lock, and the Lock must be held when testing the condition predicate and when callingawaitandsignal.[11]
[11] ReentrantLock requires that the Lock be held when calling signal or signalAll, but Lock implementations are permitted to constructConditionsthatdonothavethisrequirement.
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 189
Choose between using explicit Conditions and intrinsic condition queues in the same way as you would choose betweenReentrantLockandsynchronized:useConditionifyouneeditsadvancedfeaturessuchasfairqueueingor multiplewaitsetsperlock,andotherwisepreferintrinsicconditionqueues.(IfyoualreadyuseReentrantLockbecause youneeditsadvancedfeatures,thechoiceisalreadymade.)
14.4.AnatomyofaSynchronizer
The interfaces of ReentrantLock and Semaphore have a lot in common. Both classes act as a "gate", allowing only a limited number of threads to pass at a time; threads arrive at the gate and are allowed through (lock or acquire returnssuccessfully),aremadetowait(lockoracquireblocks),orareturnedaway(tryLockortryAcquirereturns false,indicatingthatthelockorpermitdidnotbecomeavailableinthetimeallowed).Further,bothallowinterruptible, uninterruptible,andtimedacquisitionattempts,andbothallowachoiceoffairornonfairqueueingofwaitingthreads.
Given this commonality, you might think that Semaphore was implemented on top of ReentrantLock, or perhaps ReentrantLock was implemented as a Semaphore with one permit. This would be entirely practical; it is a common exercisetoprovethatacountingsemaphorecanbeimplementedusingalock(asinSemaphoreOnLockinListing14.12) andthatalockcanbeimplementedusingacountingsemaphore.
Inactuality,theyarebothimplementedusingacommonbaseclass,Abstract-QueuedSynchronizer(AQS)asaremany other synchronizers. AQS is a framework for building locks and synchronizers, and a surprisingly broad range of synchronizerscanbebuilteasilyandefficientlyusingit.Notonlyare ReentrantLockandSemaphorebuiltusingAQS, butsoareCountDownLatch,ReentrantReadWriteLock,SynchronousQueue,[12]andFutureTask.
[12]Java6replacestheAQSͲbasedSynchronousQueuewitha(morescalable)nonͲblockingversion.
Listing14.11.BoundedBufferUsingExplicitConditionVariables.
@ThreadSafe
public class ConditionBoundedBuffer<T> {
protected final Lock lock = new ReentrantLock();
// CONDITION PREDICATE: notFull (count < items.length)
private final Condition notFull = lock.newCondition();
// CONDITION PREDICATE: notEmpty (count > 0)
private final Condition notEmpty = lock.newCondition();
@GuardedBy("lock")
private final T[] items = (T[]) new Object[BUFFER_SIZE];
@GuardedBy("lock") private int tail, head, count;
// BLOCKS-UNTIL: notFull
public void put(T x) throws InterruptedException {
lock.lock();
try {
while (count == items.length)
notFull.await();
items[tail] = x;
if (++tail == items.length)
tail = 0;
++count;
notEmpty.signal();
} finally {
lock.unlock();
}
}
// BLOCKS-UNTIL: notEmpty
public T take() throws InterruptedException {
lock.lock();
try {
while (count == 0)
notEmpty.await();
T x = items[head];
items[head] = null;
if (++head == items.length)
head = 0;
--count;
notFull.signal();
return x;
} finally {
lock.unlock();
}
}
}
190 JavaConcurrencyInPractice
Listing14.12.CountingSemaphoreImplementedUsingLock.
// Not really how java.util.concurrent.Semaphore is implemented
@ThreadSafe
public class SemaphoreOnLock {
private final Lock lock = new ReentrantLock();
// CONDITION PREDICATE: permitsAvailable (permits > 0)
private final Condition permitsAvailable = lock.newCondition();
@GuardedBy("lock") private int permits;
SemaphoreOnLock(int initialPermits) {
lock.lock();
try {
permits = initialPermits;
} finally {
lock.unlock();
}
}
// BLOCKS-UNTIL: permitsAvailable
public void acquire() throws InterruptedException {
lock.lock();
try {
while (permits <= 0)
permitsAvailable.await();
--permits;
} finally {
lock.unlock();
}
}
public void release() {
lock.lock();
try {
++permits;
permitsAvailable.signal();
} finally {
lock.unlock();
}
}
}
AQS handles many of the details of implementing a synchronizer, such as FIFO queuing of waiting threads. Individual synchronizerscandefineflexiblecriteriaforwhetherathreadshouldbeallowedtopassorberequiredtowait.
Using AQS to build synchronizers offers several benefits. Not only does it substantially reduce the implementation effort,butyoualsoneedn'tpayformultiplepointsofcontention,asyouwouldwhenconstructingonesynchronizeron topofanother.InSemaphoreOnLock,acquiringapermithastwoplaceswhereitmightblockͲonceatthelockguarding thesemaphorestate,andthenagainifapermitisnotavailable.SynchronizersbuiltwithAQShaveonlyonepointwhere theymightblock,reducingcontextͲswitchoverheadandimprovingthroughput.AQSwasdesignedforscalability,andall thesynchronizersinjava.util.concurrentthatarebuiltwithAQSbenefitfromthis.
14.5.AbstractQueuedSynchronizer
MostdeveloperswillprobablyneveruseAQSdirectly;thestandardsetofsynchronizerscoversafairlywiderangeof situations.Butseeinghowthestandardsynchronizersareimplementedcanhelpclarifyhowtheywork.
ThebasicoperationsthatanAQSͲbasedsynchronizerperformsaresomevariantsofacquireandrelease.Acquisitionis the stateͲdependent operation and can always block. With a lock or semaphore, the meaning of acquire is straightforwardͲacquirethelockorapermitͲandthecallermayhavetowaituntilthesynchronizerisinastatewhere thatcanhappen.WithCountDownLatch,acquiremeans"waituntilthelatchhasreacheditsterminalstate",andwith FutureTask, it means "wait until the task has completed". Release is not a blocking operation; a release may allow threadsblockedinacquiretoproceed.
ForaclasstobestateͲdependent,itmusthavesomestate.AQStakesonthetaskofmanagingsomeofthestateforthe synchronizer class: it manages a single integer of state information that can be manipulated through the protected getState, setState, and compareAndSetState methods. This can be used to represent arbitrary state; for example, ReentrantLockusesittorepresentthecountoftimestheowningthreadhasacquiredthelock,Semaphoreusesitto representthenumberofpermitsremaining,andFutureTaskusesittorepresentthestateofthetask(notyetstarted, running, completed, cancelled). Synchronizers can also manage additional state variables themselves; for example, ReentrantLock keeps track of the current lock owner so it can distinguish between reentrant and contended lockͲ
acquisitionrequests.
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 191
AcquisitionandreleaseinAQStaketheformsshowninListing14.13.Dependingonthesynchronizer,acquisitionmight beexclusive,aswithReentrant-Lock,ornonexclusive,aswithSemaphoreandCountDownLatch.Anacquireoperation hastwoparts.First,thesynchronizerdecideswhetherthecurrentstatepermitsacquisition;ifso,thethreadisallowed to proceed, and if not, the acquire blocks or fails. This decision is determined by the synchronizer semantics; for example,acquiringalockcansucceedifthelockisunheld,andacquiringalatchcansucceedifthelatchisinitsterminal state.
The second part involves possibly updating the synchronizer state; one thread acquiring the synchronizer can affect whetherotherthreadscanacquireit.Forexample,acquiringalockchangesthelockstatefrom"unheld"to"held",and acquiringapermitfromaSemaphorereducesthenumberofpermitsleft.Ontheotherhand,theacquisitionofalatch byonethreaddoesnotaffectwhetherotherthreadscanacquireit,soacquiringalatchdoesnotchangeitsstate.
Listing14.13.CanonicalFormsforAcquisitionandReleaseinAQS.
boolean acquire() throws InterruptedException {
while (state does not permit acquire) {
if (blocking acquisition requested) {
enqueue current thread if not already queued
block current thread
}
else
return failure
}
possibly update synchronization state
dequeue thread if it was queued
return success
}
void release() {
update synchronization state
if (new state may permit a blocked thread to acquire)
unblock one or more queued threads
}
A synchronizer supporting exclusive acquisition should implement the protected methods TRyAcquire, TRyRelease, and isHeldExclusively, and those supporting shared acquisition should implement tryAcquireShared and TRyReleaseShared.Theacquire,acquireShared,release,andreleaseSharedmethodsinAQScalltheTRyformsof thesemethodsinthesynchronizersubclasstodetermineiftheoperationcanproceed.Thesynchronizersubclasscan use getState, setState, and compareAndSetState to examine and update the state according to its acquire and releasesemantics,andinformsthebaseclassthroughthereturnstatuswhethertheattempttoacquireorreleasethe synchronizer was successful. For example, returning a negative value from TRyAcquireShared indicates acquisition failure;returningzeroindicatesthesynchronizerwasacquiredexclusively;andreturningapositivevalueindicatesthe synchronizerwasacquirednonexclusively.TheTRyReleaseandTRyReleaseSharedmethodsshouldreturntrueifthe releasemayhaveunblockedthreadsattemptingtoacquirethesynchronizer.
Tosimplifyimplementationoflocksthatsupportconditionqueues(likeReentrantLock),AQSalsoprovidesmachinery forconstructingconditionvariablesassociatedwithsynchronizers.
14.5.1.ASimpleLatch
OneShotLatchinListing14.14isabinarylatchimplementedusingAQS.Ithastwopublicmethods,awaitandsignal, thatcorrespondtoacquisitionandrelease.Initially,thelatchisclosed;anythreadcallingawaitblocksuntilthelatchis opened.Oncethelatchisopenedbyacalltosignal,waitingthreadsarereleasedandthreadsthatsubsequentlyarrive atthelatchwillbeallowedtoproceed.
192 JavaConcurrencyInPractice
Listing14.14.BinaryLatchUsingAbstractQueuedSynchronizer.
@ThreadSafe
public class OneShotLatch {
private final Sync sync = new Sync();
public void signal() { sync.releaseShared(0); }
public void await() throws InterruptedException {
sync.acquireSharedInterruptibly(0);
}
private class Sync extends AbstractQueuedSynchronizer {
protected int tryAcquireShared(int ignored) {
// Succeed if latch is open (state == 1), else fail
return (getState() == 1) ? 1 : -1;
}
protected boolean tryReleaseShared(int ignored) {
setState(1); // Latch is now open
return true; // Other threads may now be able to acquire
}
}
}
In OneShotLatch, the AQS state holds the latch state Ͳ closed (zero) or open (one). The await method calls acquireSharedInterruptibly in AQS, which in turn consults the TRyAcquireShared method in OneShotLatch. The tryAcquire-Sharedimplementationmustreturnavalueindicatingwhetherornotacquisitioncanproceed.Ifthelatch has been previously opened, tryAcquireShared returns success, allowing the thread to pass; otherwise it returns a value indicating that the acquisition attempt failed. The acquireSharedInterruptibly method interprets failure to meanthatthethreadshouldbeplacedonthequeueofwaitingthreads.Similarly,signalcallsreleaseShared,which causestryReleaseSharedtobeconsulted.TheTRyReleaseSharedimplementationunconditionallysetsthelatchstate toopenandindicates(throughitsreturnvalue)thatthesynchronizerisinafullyreleasedstate.ThiscausesAQStolet allwaitingthreadsattempttoreacquirethesynchronizer,andacquisitionwillnowsucceedbecausetryAcquireShared returnssuccess.
OneShotLatchisafullyfunctional,usable,performantsynchronizer,implementedinonlytwentyorsolinesofcode.Of course,itismissingsomeusefulfeatureͲssuchastimedacquisitionortheabilitytoinspectthelatchstatebuttheseare easy to implement as well, since AQS provides timed versions of the acquisition methods and utility methods for commoninspectionoperations.
OneShotLatchcouldhavebeenimplementedbyextendingAQSratherthandelegatingtoit,butthisisundesirablefor severalreasons[EJItem14].Doingsowouldunderminethesimple(twoͲmethod)interfaceofOneShotLatch,andwhile thepublicmethodsofAQSwon'tallowcallerstocorruptthelatchstate,callerscouldeasilyusethemincorrectly.None ofthesynchronizersinjava.util.concurrentextendsAQSdirectlyͲtheyalldelegatetoprivateinnersubclassesof AQSinstead.
14.6.AQSinJava.util.concurrentSynchronizerClasses
Many
of
the
blocking
classes
in
java.util.concurrent,
such
as
ReentrantLock,
Semaphore,
ReentrantReadWriteLock,CountDownLatch,SynchronousQueue,andFutureTask,arebuiltusingAQS.Withoutgetting toodeeplyintothedetails(thesourcecodeispartoftheJDKdownload[13]),let'stakeaquicklookathoweachofthese classesusesAQS.
[13]Orwithfewerlicensingrestrictionsathttp://gee.cs.oswego.edu/dl/concurrency-interest.
14.6.1.ReentrantLock
ReentrantLock supports only exclusive acquisition, so it implements tryAcquire, tryRelease, and isHeldExclusively; tryAcquire for the nonͲfair version is shown in Listing 14.15. ReentrantLock uses the synchronization state to hold the lock acquisition count, and maintains an owner variable holding the identity of the owningthreadthatismodifiedonlywhenthecurrentthreadhasjustacquiredthelockorisjustabouttoreleaseit.[14]In tryRelease, it checks the owner field to ensure that the current thread owns the lock before allowing an unlock to proceed;intryAcquire,itusesthisfieldtodifferentiatebetweenareentrantacquisitionandacontendedacquisition attempt.
7BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 193
[14]BecausetheprotectedstateͲmanipulationmethodshavethememorysemanticsofavolatilereadorwriteandReentrantLockiscarefulto readtheownerfieldonlyaftercallinggetStateandwriteitonlybeforecallingsetState,ReentrantLockcanpiggybackonthememory semanticsofthesynchronizationstate,andthusavoidfurthersynchronizationͲseeSection16.1.4.
Whenathreadattemptstoacquirealock,tryAcquirefirstconsultsthelockstate.Ifitisunheld,ittriestoupdatethe lockstatetoindicatethatitisheld.Becausethestatecouldhavechangedsinceitwasfirstinspectedafewinstructions ago,tryAcquireusescompareAndSetStatetoattempttoatomicallyupdatethestatetoindicatethatthelockisnow heldandconfirmthatthestatehasnotchangedsincelastobserved.(SeethedescriptionofcompareAndSetinSection 15.3.) If the lock state indicates that it is already held, if the current thread is the owner of the lock, the acquisition countisincremented;ifthecurrentthreadisnottheownerofthelock,theacquisitionattemptfails.
Listing14.15. tryAcquireImplementationFromNonǦfairReentrantLock.
protected boolean tryAcquire(int ignored) {
final Thread current = Thread.currentThread();
int c = getState();
if (c == 0) {
if (compareAndSetState(0, 1)) {
owner = current;
return true;
}
} else if (current == owner) {
setState(c+1);
return true;
}
return false;
}
ReentrantLock also takes advantage of AQS's builtͲin support for multiple condition variables and wait sets.
Lock.newConditionreturnsanewinstanceofConditionObject,aninnerclassofAQS.
14.6.2.SemaphoreandCountDownLatch
SemaphoreusestheAQSsynchronizationstatetoholdthecountofpermitscurrentlyavailable.ThetryAcquireShared method(seeListing14.16)firstcomputesthenumberofpermitsremaining,andiftherearenotenough,returnsavalue indicating that the acquire failed. If sufficient permits appear to be left, it attempts to atomically reduce the permit count using compareAndSetState. If that succeeds (meaning that the permit count had not changed since it last looked),itreturnsavalueindicatingthattheacquiresucceeded.Thereturnvaluealsoencodeswhetherothershared acquisitionattemptsmightsucceed,inwhichcaseotherwaitingthreadswillalsobeunblocked.
The while loop terminates either when there are not enough permits or when TRyAcquireShared can atomically updatethepermitcounttoreflectacquisition.WhileanygivencalltocompareAndSetStatemayfailduetocontention withanotherthread(seeSection15.3),causingittoretry,oneofthesetwoterminationcriteriawillbecometruewithin areasonablenumberofretries.Similarly,tryReleaseSharedincreasesthepermitcount,potentiallyunblockingwaiting threads,andretriesuntiltheupdatesucceeds.ThereturnvalueofTRyReleaseSharedindicateswhetherotherthreads mighthavebeenunblockedbytherelease.
CountDownLatch uses AQS in a similar manner to Semaphore: the synchronization state holds the current count. The countDown method calls release, which causes the counter to be decremented and unblocks waiting threads if the counter has reached zero; await calls acquire, which returns immediately if the counter has reached zero and otherwiseblocks.
Listing14.16. tryacquiresharedandtryreleasesharedfromSemaphore.
protected int tryAcquireShared(int acquires) {
while (true) {
int available = getState();
int remaining = available - acquires;
if (remaining < 0
|| compareAndSetState(available, remaining))
return remaining;
}
}
protected boolean tryReleaseShared(int releases) {
while (true) {
int p = getState();
if (compareAndSetState(p, p + releases))
return true;
}
}
194 JavaConcurrencyInPractice
14.6.3.FutureTask
Atfirstglance,FutureTaskdoesn'tevenlooklikeasynchronizer.ButFuture.gethassemanticsthatareverysimilarto thatofalatchifsomeevent(thecompletionorcancellationofthetaskrepresentedbytheFutureTask)hasoccurred, thenthreadscanproceed,otherwisetheyarequeueduntilthateventoccurs.
FutureTask uses the AQS synchronization state to hold the task statusrunning, completed, or cancelled. It also maintainsadditionalstatevariablestoholdtheresultofthecomputationortheexceptionitthrew.Itfurthermaintains a reference to the thread that is running the computation (if it is currently in the running state), so that it can be interruptedifthetaskiscancelled.
14.6.4.ReentrantReadWriteLock
The interface for ReadWriteLock suggests there are two locksa reader lock and a writer lockbut in the AQSͲbased implementation of ReentrantReadWriteLock, a single AQS subclass manages both read and write locking.
ReentrantRead-WriteLock uses 16 bits of the state for the writeͲlock count, and the other 16 bits for the readͲlock count.Operationsonthereadlockusethesharedacquireandreleasemethods;operationsonthewritelockusethe exclusiveacquireandreleasemethods.
Internally, AQS maintains a queue of waiting threads, keeping track of whether a thread has requested exclusive or sharedaccess.InReentrantRead-WriteLock,whenthelockbecomesavailable,ifthethreadattheheadofthequeue was looking for write access it will get it, and if the thread at the head of the queue was looking for read access, all queuedthreadsuptothefirstwriterwillgetit.[15]
[15]ThismechanismdoesnotpermitthechoiceofareaderͲpreferenceorwriterͲpreferencepolicy,assomereadͲwritelockimplementationsdo.
Forthat,eithertheAQSwaitqueuewouldneedtobesomethingotherthanaFIFOqueue,ortwoqueueswouldbeneeded.However,suchastrict orderingpolicyisrarelyneededinpractice;ifthenonfairversionofReentrantReadWriteLockdoesnotofferacceptableliveness,thefair versionusuallyprovidessatisfactoryorderingandguaranteesnonstarvationofreadersandwriters.
Summary
IfyouneedtoimplementastateͲdependentclassͲonewhosemethodsmustblockifastateͲbasedpreconditiondoes not hold Ͳ the best strategy is usually to build upon an existing library class such as Semaphore, BlockingQueue, or CountDownLatch,asinValueLatchonpage187.However,sometimesexistinglibraryclassesdonotprovideasufficient foundation;inthesecases,youcanbuildyourownsynchronizersusingintrinsicconditionqueues,explicit Condition objects, or AbstractQueuedSynchronizer. Intrinsic condition queues are tightly bound to intrinsic locking, since the mechanism for managing state dependence is necessarily tied to the mechanism for ensuring state consistency.
Similarly, explicit Conditions are tightly bound to explicit Locks, and offer an extended feature set compared to intrinsicconditionqueues,includingmultiplewaitsetsperlock,interruptibleoruninterruptibleconditionwaits,fairor nonfairqueuing,anddeadlineͲbasedwaiting.
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 195
Chapter15.AtomicVariablesandNonǦblockingSynchronization
Many of the classes in java.util.concurrent, such as Semaphore and ConcurrentLinkedQueue, provide better performanceandscalabilitythanalternativesusingsynchronized.Inthischapter,wetakealookattheprimarysource ofthisperformanceboost:atomicvariablesandnonͲblockingsynchronization.
Much of the recent research on concurrent algorithms has focused on nonͲblocking algorithms, which use lowͲlevel atomic machine instructions such as compareͲandͲswap instead of locks to ensure data integrity under concurrent access.NonͲblockingalgorithmsareusedextensivelyinoperatingsystemsandJVMsforthreadandprocessscheduling, garbagecollection,andtoimplementlocksandotherconcurrentdatastructures.
NonͲblockingalgorithmsareconsiderablymorecomplicatedtodesignandimplementthanlockͲbasedalternatives,but they can offer significant scalability and liveness advantages. They coordinate at a finer level of granularity and can greatly reduce scheduling overhead because they don't block when multiple threads contend for the same data.
Further, they are immune to deadlock and other liveness problems. In lockͲbased algorithms, other threads cannot makeprogressifathreadgoestosleeporspinswhileholdingalock,whereasnonͲblockingalgorithmsareimperviousto individualthreadfailures.AsofJava5.0,itispossibletobuildefficientnonͲblockingalgorithmsinJavausingtheatomic variableclassessuchasAtomicIntegerandAtomicReference.
Atomicvariablescanalsobeusedas"bettervolatilevariables"evenifyouarenotdevelopingnonͲblockingalgorithms.
Atomicvariablesofferthesamememorysemanticsasvolatilevariables,butwithadditionalsupportforatomicupdates Ͳ making them ideal for counters, sequence generators, and statistics gathering while offering better scalability than lockͲbasedalternatives.
15.1.DisadvantagesofLocking
Coordinating access to shared state using a consistent locking protocol ensures that whichever thread holds the lock guardingasetofvariableshasexclusiveaccessto thosevariables,andthat anychanges made tothosevariables are visibletootherthreadsthatsubsequentlyacquirethelock.
ModernJVMscanoptimizeuncontendedlockacquisitionandreleasefairlyeffectively,butifmultiplethreadsrequest thelockatthesametimetheJVMenliststhehelpoftheoperatingsystem.Ifitgetstothispoint,someunfortunate threadwillbesuspendedandhavetoberesumedlater.[1]Whenthatthreadisresumed,itmayhavetowaitforother threadstofinishtheirschedulingquantabeforeitisactuallyscheduled.Suspendingandresumingathreadhasalotof overheadandgenerallyentailsalengthyinterruption.ForlockͲbasedclasseswithfineͲgrainedoperations(suchasthe synchronizedcollectionsclasses,wheremostmethodscontainonlyafewoperations),theratioofschedulingoverhead tousefulworkcanbequitehighwhenthelockisfrequentlycontended.
[1]AsmartJVMneednotnecessarilysuspendathreadifitcontendsforalock;itcoulduseprofilingdatatodecideadaptivelybetweensuspension andspinlockingbasedonhowlongthelockhasbeenheldduringpreviousacquisitions.
Volatile variables are a lighterͲweight synchronization mechanism than locking because they do not involve context switches or thread scheduling. However, volatile variables have some limitations compared to locking: while they provide similar visibility guarantees, they cannot be used to construct atomic compound actions. This means that volatilevariablescannotbeusedwhenonevariabledependsonanother,orwhenthenewvalueofavariabledepends on its old value. This limits when volatile variables are appropriate, since they cannot be used to reliably implement commontoolssuchascountersormutexes.[2]
[2]Itistheoreticallypossible,thoughwhollyimpractical,tousethesemanticsofvolatiletoconstructmutexesandothersynchronizers;see (Raynal,1986).
For example, while the increment operation (++i) may look like an atomic operation, it is actually three distinct operationsͲfetchthecurrentvalueofthevariable,addonetoit,andthenwritetheupdatedvalueback.Inordertonot loseanupdate,theentirereadͲmodifyͲwriteoperationmustbe atomic.Sofar,theonlywaywe'veseentodothisis withlocking,asinCounteronpage56.
Counter is threadͲsafe, and in the presence of little or no contention performs just fine. But under contention, performancesuffersbecauseofcontextͲswitchoverheadandschedulingdelays.Whenlocksareheldsobriefly,being puttosleepisaharshpenaltyforaskingforthelockatthewrongtime.
Locking has a few other disadvantages. When a thread is waiting for a lock, it cannot do anything else. If a thread holding a lock is delayed (due to a page fault, scheduling delay, or the like), then no thread that needs that lock can
196 JavaConcurrencyInPractice
makeprogress.ThiscanbeaseriousproblemiftheblockedthreadisahighͲprioritythreadbutthethreadholdingthe lock is a lowerͲpriority thread Ͳ a performance hazard known as priority inversion. Even though the higherͲpriority thread should have precedence, it must wait until the lock is released, and this effectively downgrades its priority to thatofthelowerͲprioritythread.Ifathreadholdingalockispermanentlyblocked(duetoaninfiniteloop,deadlock, livelock,orotherlivenessfailure),anythreadswaitingforthatlockcannevermakeprogress.
Even ignoring these hazards, locking is simply a heavyweight mechanism for fineͲgrained operations such as incrementingacounter.ItwouldbenicetohaveafinerͲgrainedtechniqueformanagingcontentionbetweenthreadsͲ
somethinglikevolatilevariables,butofferingthepossibilityofatomicupdatesaswell.Happily,modernprocessorsoffer uspreciselysuchamechanism.
15.2.HardwareSupportforConcurrency
ExclusivelockingisapessimistictechniqueͲitassumestheworst(ifyoudon'tlockyourdoor,gremlinswillcomeinand rearrange your stuff) and doesn't proceed until you can guarantee, by acquiring the appropriate locks, that other threadswillnotinterfere.
For fineͲgrained operations, there is an alternate approach that is often more efficient Ͳ the optimistic approach, wherebyyouproceedwithanupdate,hopefulthatyoucancompleteitwithoutinterference.Thisapproachrelieson collisiondetectiontodetermineiftherehasbeeninterferencefromotherpartiesduringtheupdate,inwhichcasethe operation fails and can be retried (or not). The optimistic approach is like the old saying, "It is easier to obtain forgivenessthanpermission",where"easier"heremeans"moreefficient".
Processorsdesignedformultiprocessoroperationprovidespecialinstructionsformanagingconcurrentaccesstoshared variables. Early processors had atomic testͲandͲset, fetchͲandͲincrement, or swap instructions sufficient for implementingmutexesthatcouldinturnbeusedtoimplementmoresophisticatedconcurrentobjects.Today,nearly every modern processor has some form of atomic readͲmodifyͲwrite instruction, such as compareͲandͲswap or loadͲ
linked/storeͲconditional. Operating systems and JVMs use these instructions to implement locks and concurrent data structures,butuntilJava5.0theyhadnotbeenavailabledirectlytoJavaclasses.
15.2.1.CompareandSwap
The approach taken by most processor architectures, including IA32 and Sparc, is to implement a compareͲandͲswap (CAS) instruction. (Other processors, such as PowerPC, implement the same functionality with a pair of instructions: loadlinkedandstoreͲconditional.)CAShasthreeoperandsͲamemorylocationVonwhichtooperate,theexpectedold value A, and the new value B. CAS atomically updates V to the new value B, but only if the value in V matches the expectedoldvalueA;otherwiseitdoesnothing.Ineithercase,itreturnsthevaluecurrentlyinV.(Thevariantcalled compareͲandͲsetinsteadreturnswhethertheoperationsucceeded.)CASmeans"IthinkVshouldhavethevalueA;ifit does,putBthere,otherwisedon'tchangeitbuttellmeIwaswrong."CASisanoptimistictechniqueͲitproceedswith theupdateinthehopeofsuccess,andcandetectfailureifanotherthreadhasupdatedthevariablesinceitwaslast examined.SimulatedCASinListing15.1illustratesthesemantics(butnottheimplementationorperformance)ofCAS.
When multiple threads attempt to update the same variable simultaneously using CAS, one wins and updates the variable's value, and the rest lose. But the losers are not punished by suspension, as they could be if they failed to acquire a lock; instead, they are told that they didn't win the race this time but can try again. Because a thread that losesaCASisnotblocked,itcandecidewhetheritwantstotryagain,takesomeotherrecoveryaction,ordonothing.[3]
Thisflexibilityeliminatesmanyofthelivenesshazardsassociatedwithlocking(thoughinunusualcasescanintroduce theriskoflivelockͲseeSection10.3.3).
[3]DoingnothingmaybeaperfectlysensibleresponsetoafailedCAS;insomenonͲblockingalgorithms,suchasthelinkedqueuealgorithmin Section15.4.2,afailedCASmeansthatsomeoneelsealreadydidtheworkyouwereplanningtodo.
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 197
Listing15.1.SimulatedCASOperation.
@ThreadSafe
public class SimulatedCAS {
@GuardedBy("this") private int value;
public synchronized int get() { return value; }
public synchronized int compareAndSwap(int expectedValue,
int newValue) {
int oldValue = value;
if (oldValue == expectedValue)
value = newValue;
return oldValue;
}
public synchronized boolean compareAndSet(int expectedValue,
int newValue) {
return (expectedValue
== compareAndSwap(expectedValue, newValue));
}
}
ThetypicalpatternforusingCASisfirsttoreadthevalueAfromV,derivethenewvalueBfromA,andthenuseCASto atomically change V from A to B so long as no other thread has changed V to another value in the meantime. CAS
addresses the problem of implementing atomic readͲmodifyͲwrite sequences without locking, because it can detect interferencefromotherthreads.
15.2.2.ANonǦblockingCounter
CasCounterinListing15.2implementsathreadͲsafecounterusingCAS.Theincrementoperationfollowsthecanonical formͲfetchtheoldvalue,transformittothenewvalue(addingone),anduseCAStosetthenewvalue.IftheCASfails, theoperationisimmediatelyretried.Retryingrepeatedlyisusuallyareasonablestrategy,althoughincasesofextreme contentionitmightbedesirabletowaitorbackoffbeforeretryingtoavoidlivelock.
CasCounterdoesnotblock,thoughitmayhavetoretryseveral[4]timesifotherthreadsareupdatingthecounteratthe same time. (In practice, if all you need is a counter or sequence generator, just use AtomicInteger or AtomicLong, whichprovideatomicincrementandotherarithmeticmethods.)
[4]Theoretically,itcouldhavetoretryarbitrarilymanytimesifotherthreadskeepwinningtheCASrace;inpractice,thissortofstarvationrarely happens.
Listing15.2.NonǦblockingCounterUsingCAS.
@ThreadSafe
public class CasCounter {
private SimulatedCAS value;
public int getValue() {
return value.get();
}
public int increment() {
int v;
do {
v = value.get();
}
while (v != value.compareAndSwap(v, v + 1));
return v + 1;
}
}
At first glance, the CASͲbased counter looks as if it should perform worse than a lockͲbased counter; it has more operations and a more complicated control flow, and depends on the seemingly complicated CAS operation. But in reality,CASͲbasedcounterssignificantlyoutperformlockͲbasedcountersifthereisevenasmallamountofcontention, andoftenevenifthereisnocontention.Thefastpathforuncontendedlockacquisitiontypicallyrequiresatleastone CASplusotherlockͲrelatedhousekeeping,somoreworkisgoingoninthebestcaseforalockͲbasedcounterthaninthe normal case for the CASͲbased counter. Since the CAS succeeds most of the time (assuming low to moderate contention),thehardwarewillcorrectlypredictthebranchimplicitinthewhileloop,minimizingtheoverheadofthe morecomplicatedcontrollogic.
Thelanguagesyntaxforlockingmaybecompact,buttheworkdonebytheJVMandOStomanagelocksisnot.Locking entailstraversingarelativelycomplicatedcodepathintheJVMandmayentailOSͲlevellocking,threadsuspension,and context switches. In the best case, locking requires at least one CAS, so using locks moves the CAS out of sight but
198 JavaConcurrencyInPractice
doesn'tsaveanyactualexecutioncost.Ontheotherhand,executingaCASfromwithintheprograminvolvesnoJVM
code,systemcalls,orschedulingactivity.Whatlookslikealongercodepathattheapplicationlevelisinfactamuch shortercodepathwhenJVMandOSactivityaretakenintoaccount.TheprimarydisadvantageofCASisthatitforces the caller to deal with contention (by retrying, backing off, or giving up), whereas locks deal with contention automaticallybyblockinguntilthelockisavailable.[5]
[5]Actually,thebiggestdisadvantageofCASisthedifficultyofconstructingthesurroundingalgorithmscorrectly.
CAS performance varies widely across processors. On a singleͲCPU system, a CAS typically takes on the order of a handful of clock cycles, since no synchronization across processors is necessary. As of this writing, the cost of an uncontended CAS on multiple CPU systems ranges from about ten to about 150 cycles; CAS performance is a rapidly moving target and varies not only across architectures but even across versions of the same processor. Competitive forceswilllikelyresultincontinuedCASperformanceimprovementoverthenextseveralyears.Agoodruleofthumbis thatthecostofthe"fastpath"foruncontendedlockacquisitionandreleaseonmostprocessorsisapproximatelytwice thecostofaCAS.
15.2.3.CASSupportintheJVM
So,howdoesJavacodeconvincetheprocessortoexecuteaCASonitsbehalf?PriortoJava5.0,therewasnowaytodo thisshortofwritingnativecode.InJava5.0,lowͲlevelsupportwasaddedtoexposeCASoperationsonint,long,and objectreferences,andtheJVMcompilestheseintothemostefficientmeansprovidedbytheunderlyinghardware.On platformssupportingCAS,theruntimeinlinesthemintotheappropriatemachineinstruction(s);intheworstcase,ifa CASͲlikeinstructionisnotavailabletheJVMusesaspinlock.ThislowͲlevelJVMsupportisusedbytheatomicvariable classes (AtomicXXX in java.util.concurrent. atomic) to provide an efficient CAS operation on numeric and reference types; these atomic variable classes are used, directly or indirectly, to implement most of the classes in java.util.concurrent.
15.3.AtomicVariableClasses
Atomic variables are finerͲgrained and lighterͲweight than locks, and are critical for implementing highͲperformance concurrentcodeonmultiprocessorsystems.Atomicvariableslimitthescopeofcontentiontoasinglevariable;thisisas fineͲgrained as you can get (assuming your algorithm can even be implemented using such fine granularity). The fast (uncontended) path for updating an atomic variable is no slower than the fast path for acquiring a lock, and usually faster; the slow path is definitely faster than the slow path for locks because it does not involve suspending and reschedulingthreads.Withalgorithmsbasedonatomicvariablesinsteadoflocks,threadsaremorelikelytobeableto proceedwithoutdelayandhaveaneasiertimerecoveringiftheydoexperiencecontention.
The atomic variable classes provide a generalization of volatile variables to support atomic conditional readͲmodifyͲ
writeoperations.AtomicIntegerrepresentsanintvalue,andprovidesgetandsetmethodswiththesamememory semanticsasreadsandwritestoavolatileint.ItalsoprovidesanatomiccompareAndSetmethod(whichifsuccessful hasthe memoryeffectsofbothreadingandwritingavolatilevariable)and,forconvenience,atomicadd,increment, anddecrementmethods.AtomicIntegerbearsasuperficialresemblancetoanextendedCounterclass,butoffersfar greaterscalabilityundercontentionbecauseitcandirectlyexploitunderlyinghardwaresupportforconcurrency.
There are twelve atomic variable classes, divided into four groups: scalars, field updaters, arrays, and compound variables.Themostcommonlyusedatomicvariablesarethescalars:AtomicInteger,AtomicLong,AtomicBoolean,and AtomicReference. All support CAS; the Integer and Long versions support arithmetic as well. (To simulate atomic variables of other primitive types, you can cast short or byte values to and from int, and use floatToIntBits or doubleToLongBitsforfloatingͲpointnumbers.)
The atomic array classes (available in Integer, Long, and Reference versions) are arrays whose elements can be updatedatomically.Theatomicarrayclassesprovidevolatileaccesssemanticstotheelementsofthearray,afeature not available for ordinary arrays Ͳ a volatile array has volatile semantics only for the array reference, not for its elements.(TheothertypesofatomicvariablesarediscussedinSections15.4.3and15.4.4.) While the atomic scalar classes extend Number, they do not extend the primitive wrapper classes such as Integer or Long. In fact, they cannot: the primitive wrapper classes are immutable whereas the atomic variable classes are mutable. The atomic variable classes also do not redefine hashCode or equals; each instance is distinct. Like most mutableobjects,theyarenotgoodcandidatesforkeysinhashͲbasedcollections.
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 199
15.3.1.Atomicsas"BetterVolatiles"
InSection3.4.2,weusedavolatilereferencetoanimmutableobjecttoupdatemultiplestatevariablesatomically.
ThatexamplereliedoncheckͲthenͲact,butinthatparticularcasetheracewasharmlessbecausewedidnotcareifwe occasionally lost an update. In most other situations, such a checkͲthenͲact would not be harmless and could compromise data integrity. For example, NumberRange on page 67 could not be implemented safely with avolatile referencetoanimmutableholderobjectfortheupperandlowerbounds,norwithusingatomicintegerstostorethe bounds.Becauseaninvariantconstrainsthetwonumbersandtheycannotbeupdatedsimultaneouslywhilepreserving theinvariant,anumberrangeclassusingvolatilereferencesormultipleatomicintegerswillhaveunsafecheckͲthenͲ
actsequences.
WecancombinethetechniquefromOneValueCachewithatomicreferencestoclosetheraceconditionbyatomically updating the reference to an immutable object holding the lower and upper bounds. CasNumberRange in Listing 15.3
usesanAtomicReferencetoanIntPairtoholdthestate;byusingcompareAndSetitcanupdatetheupperorlower boundwithouttheraceconditionsofNumberRange.
Listing15.3.PreservingMultivariableInvariantsUsingCAS.
public class CasNumberRange {
@Immutable
private static class IntPair {
final int lower; // Invariant: lower <= upper
final int upper;
...
}
private final AtomicReference<IntPair> values =
new AtomicReference<IntPair>(new IntPair(0, 0));
public int getLower() { return values.get().lower; }
public int getUpper() { return values.get().upper; }
public void setLower(int i) {
while (true) {
IntPair oldv = values.get();
if (i > oldv.upper)
throw new IllegalArgumentException(
"Can't set lower to " + i + " > upper");
IntPair newv = new IntPair(i, oldv.upper);
if (values.compareAndSet(oldv, newv))
return;
}
}
// similarly for setUpper
}
15.3.2.PerformanceComparison:LocksVersusAtomicVariables
To demonstrate the differences in scalability between locks and atomic variables, we constructed a benchmark comparing several implementations of a pseudorandom number generator (PRNG). In a PRNG, the next "random"
numberisadeterministicfunctionofthepreviousnumber,soaPRNGmustrememberthepreviousnumberaspartof itsstate.
Listings15.4and15.5showtwoimplementationsofathreadͲsafePRNG,oneusingReentrantLockandtheotherusing AtomicInteger.Thetestdriverinvokeseachrepeatedly;eachiterationgeneratesarandomnumber(whichfetchesand modifiesthesharedseedstate)andalsoperformsanumberof"busyͲwork"iterationsthatoperatestrictlyonthreadͲ
localdata.Thissimulatestypicaloperationsthatincludesomeportionofoperatingonsharedstateandsomeportionof operatingonthreadͲlocalstate.
Figures15.1and15.2showthroughputwithlowandmoderatelevelsofsimulatedworkineachiteration.Withalow level of threadͲlocal computation, the lock or atomic variable experiences heavy contention; with more threadͲlocal computation,thelockoratomicvariableexperienceslesscontentionsinceitisaccessedlessoftenbyeachthread.
200 JavaConcurrencyInPractice
Figure15.1. LockandAtomicIntegerPerformanceUnderHighContention.
Figure15.2. LockandAtomicIntegerPerformanceUnderModerateContention.
Listing15.4.RandomNumberGeneratorUsingReentrantLock.
@ThreadSafe
public class ReentrantLockPseudoRandom extends PseudoRandom {
private final Lock lock = new ReentrantLock(false);
private int seed;
ReentrantLockPseudoRandom(int seed) {
this.seed = seed;
}
public int nextInt(int n) {
lock.lock();
try {
int s = seed;
seed = calculateNext(s);
int remainder = s % n;
return remainder > 0 ? remainder : remainder + n;
} finally {
lock.unlock();
}
}
}
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 201
Listing15.5.RandomNumberGeneratorUsingAtomicInteger.
@ThreadSafe
public class AtomicPseudoRandom extends PseudoRandom {
private AtomicInteger seed;
AtomicPseudoRandom(int seed) {
this.seed = new AtomicInteger(seed);
}
public int nextInt(int n) {
while (true) {
int s = seed.get();
int nextSeed = calculateNext(s);
if (seed.compareAndSet(s, nextSeed)) {
int remainder = s % n;
return remainder > 0 ? remainder : remainder + n;
}
}
}
}
As these graphs show, at high contention levels locking tends to outperform atomic variables, but at more realistic contentionlevelsatomicvariablesoutperformlocks.[6]Thisisbecausealockreactstocontentionbysuspendingthreads, reducingCPUusageandsynchronizationtrafficonthesharedmemorybus.(Thisissimilartohowblockingproducersin aproducerͲconsumerdesignreducestheloadonconsumersandtherebyletsthemcatchup.)Ontheotherhand,with atomic variables, contention management is pushed back to the calling class. Like most CASͲbased algorithms, AtomicPseudoRandom reacts to contention by trying again immediately, which is usually the right approach but in a highͲcontentionenvironmentjustcreatesmorecontention.
[6]Thesameholdstrueinotherdomains:trafficlightsprovidebetterthroughputforhightrafficbutrotariesprovidebetterthroughputforlow traffic;thecontentionschemeusedbyEthernetnetworksperformsbetteratlowtrafficlevels,butthetokenͲpassingschemeusedbytokenring networksdoesbetterwithheavytraffic.
BeforewecondemnAtomicPseudoRandomaspoorlywrittenoratomicvariablesasapoorchoicecomparedtolocks,we should realize that the level of contention in Figure 15.1 is unrealistically high: no real program does nothing but contendforalockoratomicvariable.Inpractice,atomicstendtoscalebetterthanlocksbecauseatomicsdealmore effectivelywithtypicalcontentionlevels.
The performance reversal between locks and atomics at differing levels of contention illustrates the strengths and weaknessesofeach.Withlowtomoderatecontention,atomicsofferbetterscalability;withhighcontention,locksoffer better contention avoidance. (CASͲbased algorithms also outperform lockͲbased ones on singleͲCPU systems, since a CASalwayssucceedsonasingleͲCPUsystemexceptintheunlikelycasethatathreadispreemptedinthemiddleofthe readͲmodifyͲwriteoperation.)
Figures15.1and15.2includeathirdcurve;animplementationofPseudoRandomthatusesaThreadLocalforthePRNG
state. This implementation approach changes the behavior of the classeach thread sees its own private sequence of pseudorandomnumbers,insteadofallthreadssharingonesequencebutillustratesthatitisoftencheapertonotshare state at all if it can be avoided. We can improve scalability by dealing more effectively with contention, but true scalabilityisachievedonlybyeliminatingcontentionentirely.
15.4.NonǦblockingAlgorithms
LockͲbasedalgorithmsareatriskforanumberoflivenessfailures.Ifathreadholdingalockisdelayedduetoblocking I/O, page fault, or other delay, it is possible that no thread will make progress. An algorithm is called nonͲblocking if failureorsuspensionofanythreadcannotcausefailureorsuspensionofanotherthread;analgorithmiscalledlockͲfree if,ateachstep,somethreadcanmakeprogress.AlgorithmsthatuseCASexclusivelyforcoordinationbetweenthreads can,ifconstructedcorrectly,bebothnonͲblockingandlockͲfree.AnuncontendedCASalwayssucceeds,andifmultiple threadscontendforaCAS,onealwayswinsandthereforemakesprogress.NonͲblockingalgorithmsarealsoimmuneto deadlockorpriorityinversion(thoughtheycanexhibitstarvationorlivelockbecausetheycaninvolverepeatedretries).
We've seen one nonͲblocking algorithm so far: CasCounter. Good nonͲblocking algorithms are known for many common data structures, including stacks, queues, priority queues, and hash tables Ͳ though designing new ones is a taskbestlefttoexperts.
15.4.1.ANonǦblockingStack
NonͲblockingalgorithmsareconsiderablymorecomplicatedthantheirlockͲbasedequivalents.ThekeytocreatingnonͲ
blockingalgorithmsisfiguringouthowtolimitthescopeofatomicchangestoasinglevariablewhilemaintainingdata
202 JavaConcurrencyInPractice
consistency. In linked collection classes such as queues, you can sometimes get away with expressing state transformations as changes to individual links and using an AtomicReference to represent each link that must be updatedatomically.
Stacks are the simplest linked data structure: each element refers to only one other element and each element is referredtobyonlyoneobjectreference.ConcurrentStackinListing15.6showshowtoconstructastackusingatomic references.ThestackisalinkedlistofNodeelements,rootedattop,eachofwhichcontainsavalueandalinktothe nextelement.Thepushmethodpreparesanewlinknodewhosenextfieldreferstothecurrenttopofthestack,and thenusesCAStotrytoinstallitonthetopofthestack.Ifthesamenodeisstillonthetopofthestackaswhenwe started,theCASsucceeds;ifthetopnodehaschanged(becauseanotherthreadhasaddedorremovedelementssince we started), the CAS fails and push updates the new node based on the current stack state and tries again. In either case,thestackisstillinaconsistentstateaftertheCAS.
CasCounter and ConcurrentStack illustrate characteristics of all nonͲblocking algorithms: some work is done speculatively and may have to be redone. In ConcurrentStack, when we construct the Node representing the new element,wearehopingthatthevalueofthenextreferencewillstillbecorrectbythetimeitisinstalledonthestack, butarepreparedtoretryintheeventofcontention.
NonͲblocking algorithms like ConcurrentStack derive their thread safety from the fact that, like locking, compareAndSetprovidesbothatomicityandvisibilityguarantees.Whenathreadchangesthestateofthestack,itdoes sowithacompareAndSet,whichhasthememoryeffectsofavolatilewrite.Whenathreadexaminesthestack,itdoes sobycallinggetonthesameAtomicReference,whichhasthememoryeffectsofavolatileread.Soanychangesmade byonethreadaresafelypublishedtoanyotherthreadthatexaminesthestateofthelist.Andthelistismodifiedwitha compareAndSetthatatomicallyeitherupdatesthetopreferenceorfailsifitdetectsinterferencefromanotherthread.
15.4.2.ANonǦblockingLinkedList
ThetwononͲblockingalgorithmswe'veseensofar,thecounterandthestack,illustratethebasicpatternofusingCAS
toupdateavaluespeculatively,retryingiftheupdatefails.ThetricktobuildingnonͲblockingalgorithmsistolimitthe scopeofatomicchangestoasinglevariable.Withcountersthisistrivial,andwithastackitisstraightforwardenough, butformorecomplicateddatastructuressuchasqueues,hashtables,ortrees,itcangetalottrickier.
Alinkedqueueismorecomplicatedthanastackbecauseitmustsupportfastaccesstoboththeheadandthetail.Todo this, it maintains separate head and tail pointers. Two pointers refer to the node at the tail: the next pointer of the currentlastelement,andthetailpointer.Toinsertanewelementsuccessfully,bothofthesepointersmustbeupdated atomically.Atfirstglance,thiscannotbedonewithatomicvariables;separateCASoperationsarerequiredtoupdate thetwopointers,andifthefirstsucceedsbutthesecondonefailsthequeueisleftinaninconsistentstate.And,evenif both operations succeed, another thread could try to access the queue between the first and the second. Building a nonͲblockingalgorithmforalinkedqueuerequiresaplanforboththesesituations.
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 203
Listing15.6.NonǦblockingStackUsingTreiber'sAlgorithm(Treiber,1986).
@ThreadSafe
public class ConcurrentStack <E> {
AtomicReference<Node<E>> top = new AtomicReference<Node<E>>(); public void push(E item) {
Node<E> newHead = new Node<E>(item);
Node<E> oldHead;
do {
oldHead = top.get();
newHead.next = oldHead;
} while (!top.compareAndSet(oldHead, newHead));
}
public E pop() {
Node<E> oldHead;
Node<E> newHead;
do {
oldHead = top.get();
if (oldHead == null)
return null;
newHead = oldHead.next;
} while (!top.compareAndSet(oldHead, newHead));
return oldHead.item;
}
private static class Node <E> {
public final E item;
public Node<E> next;
public Node(E item) {
this.item = item;
}
}
}
Weneedseveraltrickstodevelopthisplan.Thefirstistoensurethatthedatastructureisalwaysinaconsistentstate, eveninthemiddleofanmultiͲstepupdate.Thatway,ifthreadAisinthemiddleofaupdatewhenthreadBarriveson thescene,Bcantellthatanoperationhasbeenpartiallycompletedandknowsnottotryimmediatelytoapplyitsown update.ThenBcanwait(byrepeatedlyexaminingthequeuestate)untilAfinishes,sothatthetwodon'tgetineach other'sway.
Whilethistrickbyitselfwouldsufficetoletthreads"taketurns"accessingthedatastructurewithoutcorruptingit,if onethreadfailedinthemiddleofanupdate,nothreadwouldbeabletoaccessthequeueatall.Tomakethealgorithm nonͲblocking,wemustensurethatthefailureofathreaddoesnotpreventotherthreadsfrommakingprogress.Thus, the second trick is to make sure that if B arrives to find the data structure in the middle of an update by A, enough information is already embodied in the data structure for B to finish the update for A. If B "helps" A by finishing A's operation,BcanproceedwithitsownoperationwithoutwaitingforA.WhenAgetsaroundtofinishingitsoperation,it willfindthatBalreadydidthejobforit.
LinkedQueue in Listing 15.7 shows the insertion portion of the MichaelͲScott nonͲblocking linkedͲqueue algorithm (MichaelandScott,1996),whichisusedbyConcurrentLinkedQueue.Asinmanyqueuealgorithms,anemptyqueue consistsofa"sentinel"or"dummy"node,andtheheadandtailpointersareinitializedtorefertothesentinel.Thetail pointer always refers to the sentinel (if the queue is empty), the last element in the queue, or (in the case that an operationisinmidͲupdate)thesecondͲtoͲlastelement.Figure15.3illustratesaqueuewithtwoelementsinthenormal, orquiescent,state.
Figure15.3.QueuewithTwoElementsinQuiescentState.
Insertinganewelementinvolvesupdatingtwopointers.Thefirstlinksthenewnodetotheendofthelistbyupdating thenextpointerofthecurrentlastelement;thesecondswingsthetailpointeraroundtopointtothenewlastelement.
204 JavaConcurrencyInPractice
Betweenthesetwooperations,thequeueisintheintermediatestate,showninFigure15.4.Afterthesecondupdate, thequeueisagaininthequiescentstate,showninFigure15.5.
Figure15.4.QueueinIntermediateStateDuringInsertion.
Figure15.5.QueueAgaininQuiescentStateAfterInsertionisComplete.
Thekeyobservationthatenablesbothoftherequiredtricksisthatifthequeueisinthequiescentstate,thenextfield ofthelinknodepointedtobytailisnull,andifitisintheintermediatestate,tail.nextisnonͲnull.Soanythreadcan immediatelytellthestateofthequeuebyexaminingtail.next.Further,ifthequeueisintheintermediatestate,itcan berestoredtothequiescentstatebyadvancingthetailpointerforwardonenode,finishingtheoperationforwhichever threadisinthemiddleofinsertinganelement.[7]
[7]Forafullaccountofthecorrectnessofthisalgorithm,see(MichaelandScott,1996)or(HerlihyandShavit,2006).
LinkedQueue.putfirstcheckstoseeifthequeueisintheintermediatestatebeforeattemptingtoinsertanewelement (stepA).Ifitis,thensomeotherthreadisalreadyintheprocessofinsertinganelement(betweenitsstepsCandD).
Ratherthanwaitforthatthreadtofinish,thecurrentthreadhelpsitbyfinishingtheoperationforit,advancingthetail pointer(stepB).Itthenrepeatsthischeckincaseanotherthreadhasstartedinsertinganewelement,advancingthetail pointeruntilitfindsthequeueinthequiescentstatesoitcanbeginitsowninsertion.
TheCASatstepC,whichlinksthenewnodeatthetailofthequeue,couldfailiftwothreadstrytoinsertanelementat thesametime.Inthatcase,noharmisdone:nochangeshavebeenmade,andthecurrentthreadcanjustreloadthe tailpointerandtryagain.OnceCsucceeds,theinsertionisconsideredtohavetakeneffect;thesecondCAS(stepD)is considered "cleanup", since it can be performed either by the inserting thread or by any other thread. If D fails, the insertingthreadreturnsanywayratherthanretryingtheCAS,becausenoretryisneededͲanotherthreadhasalready finished the job in its step B! This works because before any thread tries to link a new node into the queue, it first checkstoseeifthequeueneedscleaningupbycheckingiftail.nextisnonͲnull.Ifitis,itadvancesthetailpointerfirst (perhapsmultipletimes)untilthequeueisinthequiescentstate.
7BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 205
Listing15.7.InsertionintheMichaelǦScottNonǦblockingQueueAlgorithm(MichaelandScott,1996).
@ThreadSafe
public class LinkedQueue <E> {
private static class Node <E> {
final E item;
final AtomicReference<Node<E>> next;
public Node(E item, Node<E> next) {
this.item = item;
this.next = new AtomicReference<Node<E>>(next);
}
}
private final Node<E> dummy = new Node<E>(null, null); private final AtomicReference<Node<E>> head
= new AtomicReference<Node<E>>(dummy);
private final AtomicReference<Node<E>> tail
= new AtomicReference<Node<E>>(dummy);
public boolean put(E item) {
Node<E> newNode = new Node<E>(item, null);
while (true) {
Node<E> curTail = tail.get();
Node<E> tailNext = curTail.next.get();
if (curTail == tail.get()) {
if (tailNext != null) {
// Queue in intermediate state, advance tail
tail.compareAndSet(curTail, tailNext);
} else {
// In quiescent state, try inserting new node
if (curTail.next.compareAndSet(null, newNode)) {
// Insertion succeeded, try advancing tail
tail.compareAndSet(curTail, newNode);
return true;
}
}
}
}
}
}
15.4.3.AtomicFieldUpdaters
Listing15.7illustratesthealgorithmusedbyConcurrentLinkedQueue,buttheactualimplementationisabitdifferent.
Instead of representing each Node with an atomic reference, ConcurrentLinkedQueue uses an ordinary volatile referenceandupdatesitthroughthereflectionͲbasedAtomicReferenceFieldUpdater,asshowninListing15.8.
Listing15.8.UsingAtomicFieldUpdatersinConcurrentLinkedQueue.
private class Node<E> {
private final E item;
private volatile Node<E> next;
public Node(E item) {
this.item = item;
}
}
private static AtomicReferenceFieldUpdater<Node, Node> nextUpdater
= AtomicReferenceFieldUpdater.newUpdater(
Node.class, Node.class, "next");
The atomic field updater classes (available in Integer, Long, and Reference versions) represent a reflectionͲbased
"view" of an existing volatile field so that CAS can be used on existing volatile fields. The updater classes have no constructors; to create one, you call the newUpdater factory method, specifying the class and field name. The field updater classesarenottiedtoaspecificinstance; onecanbeusedtoupdatethetarget fieldforanyinstanceofthe targetclass.Theatomicityguaranteesfortheupdaterclassesareweakerthanfortheregularatomicclassesbecause you cannot guarantee that the underlying fields will not be modified directlythe compareAndSet and arithmetic methodsguaranteeatomicityonlywithrespecttootherthreadsusingtheatomicfieldupdatermethods.
In ConcurrentLinkedQueue, updates to the next field of a Node are applied using the compareAndSet method of nextUpdater. This somewhat circuitous approach is used entirely for performance reasons. For frequently allocated, shortͲlivedobjectslike queuelink nodes,eliminatingthecreationofan AtomicReferenceforeach Nodeissignificant
206 JavaConcurrencyInPractice
enoughtoreducethecostofinsertionoperations.However,innearlyallsituations,ordinaryatomicvariablesperform justfineͲinonlyafewcaseswilltheatomicfieldupdatersbeneeded.(Theatomicfieldupdatersarealsousefulwhen youwanttoperformatomicupdateswhilepreservingtheserializedformofanexistingclass.) 15.4.4.TheABAProblem
TheABAproblemisananomalythatcanarisefromthenaiveuseofcompareͲandͲswapinalgorithmswherenodescan berecycled(primarilyinenvironmentswithoutgarbagecollection).ACASeffectivelyasks"IsthevalueofVstillA?",and proceeds with the update if so. In most situations, including the examples presented in this chapter, this is entirely sufficient.However,sometimeswereallywanttoask"HasthevalueofVchangedsinceIlastobservedittobeA?"For some algorithms, changing V from A to B and then back to A still counts as a change that requires us to retry some algorithmicstep.
ThisABAproblemcanariseinalgorithmsthatdotheirownmemorymanagementforlinknodeobjects.Inthiscase,that theheadofaliststillreferstoapreviouslyobservednodeisnotenoughtoimplythatthecontentsofthelisthavenot changed.IfyoucannotavoidtheABAproblembylettingthegarbagecollectormanagelinknodesforyou,thereisstilla relatively simple solution: instead of updating the value of a reference, update a pair of values, a reference and a version number. Even if the value changes from A to B and back to A, the version numbers will be different.
AtomicStampedReference (and its cousin AtomicMarkableReference) provide atomic conditional update on a pair of variables.AtomicStampedReferenceupdatesanobjectreferenceͲintegerpair,allowing"versioned"referencesthatare immune[8]totheABAproblem.Similarly,AtomicMarkableReferenceupdatesanobjectreferenceͲbooleanpairthatis usedbysomealgorithmstoletanoderemaininalistwhilebeingmarkedasdeleted.[9]
[8]Inpractice,anyway;theoreticallythecountercouldwrap.
[9] Many processors provide a doubleͲwide CAS (CAS2 or CASX) operation that can operate on a pointerͲinteger pair, which would make this operationreasonablyefficient.AsofJava6,Atomic-StampedReferencedoesnotusedoubleͲwideCASevenonplatformsthatsupportit.
(DoubleͲwide CAS differs from DCAS, which operates on two unrelated memory locations; as of this writing, no current processor implements DCAS.)
Summary
NonͲblocking algorithms maintain thread safety by using lowͲlevel concurrency primitives such as compareͲandͲswap insteadoflocks.TheselowͲlevelprimitivesareexposedthroughtheatomicvariableclasses,whichcanalsobeusedas
"bettervolatilevariables"providingatomicupdateoperationsforintegersandobjectreferences.
NonͲblockingalgorithmsaredifficulttodesignandimplement,butcanofferbetterscalabilityundertypicalconditions andgreaterresistancetolivenessfailures.ManyoftheadvancesinconcurrentperformancefromoneJVMversionto thenextcomefromtheuseofnonͲblockingalgorithms,bothwithintheJVMandintheplatformlibraries.
7BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel
207
Chapter16.TheJavaMemoryModel
Throughoutthisbook,we'vemostlyavoidedthelowͲleveldetailsoftheJavaMemoryModel(JMM)andinsteadfocused onhigherͲleveldesignissuessuchassafepublication,specificationof,andadherencetosynchronizationpolicies.These derivetheirsafetyfromtheJMM,andyoumayfinditeasiertousethesemechanismseffectivelywhenyouunderstand why they work. This chapter pulls back the curtain to reveal the lowͲlevel requirements and guarantees of the Java MemoryModelandthereasoningbehindsomeofthehigherͲleveldesignrulesofferedinthisbook.
16.1.WhatisaMemoryModel,andWhywouldIWantOne?
SupposeonethreadassignsavaluetoaVariable:
aVariable = 3;
Amemorymodeladdressesthequestion"UnderwhatconditionsdoesathreadthatreadsaVariableseethevalue3?"
Thismaysoundlikeadumbquestion,butintheabsenceofsynchronization,thereareanumberofreasonsathread mightnotimmediatelyͲoreverͲseetheresultsofanoperationinanotherthread.Compilersmaygenerateinstructions inadifferentorderthanthe"obvious"onesuggestedbythesourcecode,orstorevariablesinregistersinsteadofin memory;processorsmayexecuteinstructionsinparalleloroutoforder;cachesmayvarytheorderinwhichwritesto variables are committed to main memory; and values stored in processorͲlocal caches may not be visible to other processors. These factors can prevent a thread from seeing the most upͲtoͲdate value for a variable and can cause memoryactionsinotherthreadstoappeartohappenoutoforderͲifyoudon'tuseadequatesynchronization.
InasingleͲthreadedenvironment,allthesetricksplayedonourprogrambytheenvironmentarehiddenfromusand havenoeffectotherthantospeedupexecution.TheJavaLanguageSpecificationrequirestheJVMtomaintainwithin threadasͲifͲserialsemantics:aslongastheprogramhasthesameresultasifitwereexecutedinprogramorderina strictly sequential environment, all these games are permissible. And that's a good thing, too, because these rearrangements are responsible for much of the improvement in computing performance in recent years. Certainly higherclockrateshavecontributedtoimprovedperformance,butsohasincreasedparallelismͲpipelinedsuperscalar executionunits,dynamicinstructionscheduling,speculativeexecution,andsophisticatedmultilevelmemorycaches.As processors have become more sophisticated, so too have compilers, rearranging instructions to facilitate optimal execution and using sophisticated global registerͲallocation algorithms. And as processor manufacturers transition to multicoreprocessors,largelybecauseclockratesaregettinghardertoincreaseeconomically,hardwareparallelismwill onlyincrease.
In a multithreaded environment, the illusion of sequentiality cannot be maintained without significant performance cost.Sincemostofthetimethreadswithinaconcurrentapplicationareeach"doingtheirownthing",excessiveinterͲ
threadcoordinationwouldonlyslowdowntheapplicationtonorealbenefit.Itisonlywhenmultiplethreadssharedata thatitisnecessarytocoordinatetheiractivities,andtheJVMreliesontheprogramtoidentifywhenthisishappening byusingsynchronization.
TheJMMspecifiestheminimalguaranteestheJVMmustmakeaboutwhenwritestovariablesbecomevisibletoother threads.Itwasdesignedtobalancetheneedforpredictabilityandeaseofprogramdevelopmentwiththerealitiesof implementing highͲperformance JVMs on a wide range of popular processorarchitectures. Some aspects of the JMM
maybedisturbingatfirstifyouarenotfamiliarwiththetricksusedbymodernprocessorsandcompilerstosqueeze extraperformanceoutofyourprogram.
16.1.1.PlatformMemoryModels
InasharedͲmemorymultiprocessorarchitecture,eachprocessorhasitsowncachethatisperiodicallyreconciledwith mainmemory.Processorarchitecturesprovidevaryingdegreesofcachecoherence;someprovideminimalguarantees thatallowdifferentprocessorstoseedifferentvaluesforthesamememorylocationatvirtuallyanytime.Theoperating system, compiler, and runtime (and sometimes, the program, too) must make up the difference between what the hardwareprovidesandwhatthreadsafetyrequires.
Ensuringthateveryprocessorknowswhateveryotherprocessorisdoingatalltimesisexpensive.Mostofthetimethis information is not needed, so processors relax their memoryͲcoherency guarantees to improve performance. An architecture'smemorymodeltellsprogramswhatguaranteestheycanexpectfromthememorysystem,andspecifies the special instructions required (called memory barriers or fences) to get the additional memory coordination guarantees required when sharing data. In order to shield the Java developer from the differences between memory
208 JavaConcurrencyInPractice
modelsacrossarchitectures,Javaprovidesitsownmemorymodel,andtheJVMdealswiththedifferencesbetweenthe JMMandtheunderlyingplatform'smemorymodelbyinsertingmemorybarriersattheappropriateplaces.
Oneconvenientmentalmodelforprogramexecutionistoimaginethatthereisasingleorderinwhichtheoperations happeninaprogram,regardlessofwhatprocessortheyexecuteon,andthateachreadofavariablewillseethelast write in the execution order to that variable by any processor. This happy, if unrealistic, model is called sequential consistency.Softwaredevelopersoftenmistakenlyassumesequentialconsistency,butnomodernmultiprocessoroffers sequentialconsistencyandtheJMMdoesnoteither.Theclassicsequentialcomputingmodel,thevonNeumannmodel, isonlyavagueapproximationofhowmodernmultiprocessorsbehave.
The bottom line is that modern sharedͲmemory multiprocessors (and compilers) can do some surprising things when data is shared across threads, unless you've told them not to through the use of memory barriers. Fortunately, Java programs need not specify the placement of memory barriers; they need only identify when shared state is being accessed,throughtheproperuseofsynchronization.
16.1.2.Reordering
In describing race conditions and atomicity failures in Chapter 2, we used interaction diagrams depicting "unlucky timing" where the scheduler interleaved operations so as to cause incorrect results in insufficiently synchronized programs. To make matters worse, the JMM can permit actions to appear to execute in different orders from the perspective of different threads, making reasoning about ordering in the absence of synchronization even more complicated.Thevariousreasonswhyoperationsmightbedelayedorappeartoexecuteoutofordercanallbegrouped intothegeneralcategoryofreordering.
PossibleReordering in Listing 16.1 illustrates how difficult it is to reason about the behavior of even the simplest concurrentprogramsunlesstheyarecorrectlysynchronized.ItisfairlyeasytoimaginehowPossibleReorderingcould print (1,0),or(0,1),or (1,1):thread Acouldrun tocompletionbeforeBstarts,Bcouldrunto completionbeforeA starts, or their actions could be interleaved. But, strangely, PossibleReordering can also print (0, 0)! The actions in eachthreadhavenodataflowdependenceoneachother,andaccordinglycanbeexecutedoutoforder.(Evenifthey areexecutedinorder,thetimingbywhichcachesareflushedtomainmemorycanmakeitappear,fromtheperspective of B, that the assignments in A occurred in the opposite order.) Figure 16.1 shows a possible interleaving with reorderingthatresultsinprinting(0,0).
Figure16.1.InterleavingShowingReorderinginPossibleReordering.
PossibleReorderingisatrivialprogram,anditisstillsurprisinglytrickytoenumerateitspossibleresults.Reorderingat thememorylevelcanmakeprogramsbehaveunexpectedly.Itisprohibitivelydifficulttoreasonaboutorderinginthe absence of synchronization; it is much easier to ensure that your program uses synchronization appropriately.
Synchronizationinhibitsthecompiler,runtime,andhardwarefromreorderingmemoryoperationsinwaysthatwould violatethevisibilityguaranteesprovidedbytheJMM.[1]
[1]Onmostpopularprocessorarchitectures,thememorymodelisstrongenoughthattheperformancecostofavolatilereadisinlinewiththatof anonvolatileread.
16.1.3.TheJavaMemoryModelin500WordsorLess
TheJavaMemoryModelisspecifiedintermsofactions,whichincludereadsandwritestovariables,locksandunlocks ofmonitors,andstartingandjoiningwiththreads.TheJMMdefinesapartialordering[2]calledhappensͲbeforeonall actionswithintheprogram.ToguaranteethatthethreadexecutingactionBcanseetheresultsofactionA(whetheror notAandBoccurindifferentthreads),theremustbeahappensͲbeforerelationshipbetweenAandB.Intheabsenceof ahappensͲbeforeorderingbetweentwooperations,theJVMisfreetoreorderthemasitpleases.
[2]Apartialordering isarelationonasetthatisantisymmetric,reflexive,andtransitive,butforanytwoelementsxandy,itneednotbethe casethatx
yory
x.Weusepartialorderingseverydaytoexpresspreferences;wemayprefersushitocheeseburgersandMozarttoMahler, butwedon'tnecessarilyhaveaclearpreferencebetweencheeseburgersandMozart.
7BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel
209
Listing16.1.InsufficientlySynchronizedProgramthatcanhaveSurprisingResults.
public class PossibleReordering {
static int x = 0, y = 0;
static int a = 0, b = 0;
public static void main(String[] args)
throws InterruptedException {
Thread one = new Thread(new Runnable() {
public void run() {
a = 1;
x = b;
}
});
Thread other = new Thread(new Runnable() {
public void run() {
b = 1;
y = a;
}
});
one.start(); other.start();
one.join(); other.join();
System.out.println("( "+ x + "," + y + ")");
}
}
Adataraceoccurswhenavariableisreadbymorethanonethread,andwrittenbyatleastonethread,butthereads andwritesarenotorderedbyhappensͲbefore.Acorrectlysynchronizedprogramisonewithnodataraces;correctly synchronizedprogramsexhibitsequentialconsistency,meaningthatallactionswithintheprogramappeartohappenin afixed,globalorder.
TherulesforhappensͲbeforeare:
x Program order rule. Each action in a thread happensͲbefore every action in that thread that comes later in the programorder.
x Monitorlockrule.AnunlockonamonitorlockhappensͲbeforeeverysubsequentlockonthatsamemonitorlock.[3]
x Volatilevariablerule.AwritetoavolatilefieldhappensͲbeforeeverysubsequentreadofthatsamefield.[4]
x Threadstartrule.AcalltoThread.startonathreadhappensͲbeforeeveryactioninthestartedthread.
x Thread termination rule. Any action in a thread happensͲbefore any other thread detects that thread has terminated,eitherbysuccessfullyreturnfromThread.joinorbyThread.isAlivereturningfalse.
x Interruptionrule.AthreadcallinginterruptonanotherthreadhappensͲbeforetheinterruptedthreaddetectsthe interrupt(eitherbyhavingInterruptedExceptionthrown,orinvokingisInterruptedorinterrupted).
x Finalizerrule.TheendofaconstructorforanobjecthappensͲbeforethestartofthefinalizerforthatobject.
x Transitivity.IfAhappensͲbeforeB,andBhappensͲbeforeC,thenAhappensͲbeforeC.
[3]LocksandunlocksonexplicitLockobjectshavethesamememorysemanticsasintrinsiclocks.
[4]Readsandwritesofatomicvariableshavethesamememorysemanticsasvolatilevariables.
Even though actions are only partially ordered, synchronization actions Ͳ lock acquisition and release, and reads and writes of volatile variables Ͳ are totally ordered. This makes it sensible to describe happensͲbefore in terms of
"subsequent"lockacquisitionsandreadsofvolatilevariables.
Figure16.2illustratesthehappensͲbeforerelationwhentwothreadssynchronizeusingacommonlock.Alltheactions withinthreadAareorderedbytheprogramorderrule,asaretheactionswithinthreadB.BecauseAreleaseslockM
andBsubsequentlyacquiresM,alltheactionsinAbeforereleasingthelockarethereforeorderedbeforetheactionsin Bafteracquiringthelock.Whentwothreadssynchronizeondifferentlocks,wecan'tsayanythingabouttheorderingof actionsbetweenthemͲthereisnohappensͲbeforerelationbetweentheactionsinthetwothreads.
210 JavaConcurrencyInPractice
Figure16.2.IllustrationofHappensǦbeforeintheJavaMemoryModel.
16.1.4.PiggybackingonSynchronization
BecauseofthestrengthofthehappensͲbeforeordering,youcansometimespiggybackonthevisibilitypropertiesofan existing synchronization. This entails combining the program order rule for happensͲbefore with one of the other orderingrules(usuallythemonitorlockorvolatilevariablerule)toorderaccessestoavariablenototherwiseguarded byalock.Thistechniqueisverysensitivetotheorderinwhichstatementsoccurandisthereforequitefragile;itisan advancedtechniquethatshouldbereservedforsqueezingthelastdropofperformanceoutofthemostperformanceͲ
criticalclasseslikeReentrantLock.
TheimplementationoftheprotectedAbstractQueuedSynchronizermethodsinFutureTaskillustratespiggybacking.
AQS maintains an integer of synchronizer state that FutureTask uses to store the task state: running, completed, or cancelled.ButFutureTaskalsomaintainsadditionalvariables,suchastheresultofthecomputation.Whenonethread callssettosavetheresultandanotherthreadcallsgettoretrieveit,thetwohadbetterbeorderedbyhappensͲbefore.
Thiscouldbedonebymakingthereferencetotheresultvolatile,butitispossibletoexploitexistingsynchronization toachievethesameresultatlowercost.
FutureTask is carefully crafted to ensure that a successful call to tryReleaseShared always happensͲbefore a subsequent call to TRyAcquireShared; try-ReleaseShared always writes to a volatile variable that is read by TRyAcquire-Shared.Listing16.2showstheinnerSetandinnerGetmethodsthatarecalledwhentheresultissavedor retrieved;sinceinnerSetwritesresultbeforecallingreleaseShared(whichcallstryReleaseShared)andinnerGet readsresultaftercallingacquireShared(whichcallsTRyAcquireShared),theprogramorderrulecombineswiththe volatilevariableruletoensurethatthewriteofresultininnerGethappensͲbeforethereadofresultininnerGet.
7BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel
211
Listing16.2.InnerClassofFutureTaskIllustratingSynchronizationPiggybacking.
// Inner class of FutureTask
private final class Sync extends AbstractQueuedSynchronizer {
private static final int RUNNING = 1, RAN = 2, CANCELLED = 4;
private V result;
private Exception exception;
void innerSet(V v) {
while (true) {
int s = getState();
if (ranOrCancelled(s))
return;
if (compareAndSetState(s, RAN))
break;
}
result = v;
releaseShared(0);
done();
}
V innerGet() throws InterruptedException, ExecutionException {
acquireSharedInterruptibly(0);
if (getState() == CANCELLED)
throw new CancellationException();
if (exception != null)
throw new ExecutionException(exception);
return result;
}
}
Wecallthistechnique"piggybacking"becauseitusesanexistinghappensͲbeforeorderingthatwascreatedforsome other reason to ensure the visibility of object X, rather than creating a happensͲbefore ordering specifically for publishingX.
PiggybackingofthesortemployedbyFutureTaskisquitefragileandshouldnotbeundertakencasually.However,in somecasespiggybackingisperfectlyreasonable,suchaswhenaclasscommitstoahappensͲbeforeorderingbetween methodsaspartofitsspecification.Forexample,safepublicationusingaBlockingQueueisaformofpiggybacking.One threadputtinganobjectonaqueueandanotherthreadsubsequentlyretrievingitconstitutessafepublicationbecause there is guaranteed to be sufficient internal synchronization in a BlockingQueue implementation to ensure that the enqueuehappensͲbeforethedequeue.
OtherhappensͲbeforeorderingsguaranteedbytheclasslibraryinclude:
x Placing an item in a threadͲsafe collection happensͲbefore another thread retrieves that item from the collection;
x CountingdownonaCountDownLatchhappensͲbeforeathreadreturnsfromawaitonthatlatch; x ReleasingapermittoaSemaphorehappensͲbeforeacquiringapermitfromthatsameSemaphore; x ActionstakenbythetaskrepresentedbyaFuturehappensͲbeforeanotherthreadsuccessfullyreturnsfrom Future.get;
x SubmittingaRunnableorCallabletoanExecutorhappensͲbeforethetaskbeginsexecution;and x AthreadarrivingataCyclicBarrierorExchangerhappensͲbeforetheotherthreadsarereleasedfromthat samebarrierorexchangepoint.IfCyclicBarrierusesabarrieraction,arrivingatthebarrierhappensͲbefore thebarrieraction,whichinturnhappensͲbeforethreadsarereleasedfromthebarrier.
16.2.Publication
Chapter3exploredhowanobjectcouldbesafelyorimproperlypublished.Thesafepublicationtechniquesdescribed therederivetheirsafetyfromguaranteesprovidedbytheJMM;therisksofimproperpublicationareconsequencesof theabsenceofahappensͲbeforeorderingbetweenpublishingasharedobjectandaccessingitfromanotherthread.
16.2.1.UnsafePublication
ThepossibilityofreorderingintheabsenceofahappensͲbeforerelationshipexplainswhypublishinganobjectwithout adequatesynchronizationcanallowanotherthreadtoseeapartiallyconstructedobject(seeSection3.5).Initializinga new object involves writing to variables Ͳ the new object's fields. Similarly, publishing a reference involves writing to anothervariableͲthereferencetothenewobject.IfyoudonotensurethatpublishingthesharedreferencehappensͲ
212 JavaConcurrencyInPractice
beforeanotherthreadloadsthatsharedreference,thenthewriteofthereferencetothenewobjectcanbereordered (from the perspective of the thread consuming the object) with the writes to its fields. In that case, another thread could see an upͲtoͲdate value for the object reference but outͲofͲdate values for some or all of that object's statea partiallyconstructedobject.
Unsafepublicationcanhappenasaresultofanincorrectlazyinitialization,asshowninFigure16.3.Atfirstglance,the onlyproblemhereseemstobetheraceconditiondescribedinSection2.2.2.Undercertaincircumstances,suchaswhen allinstancesoftheResourceareidentical,youmightbewillingtooverlookthese(alongwiththeinefficiencyofpossibly creating the Resource more than once). Unfortunately, even if these defects are overlooked, UnsafeLazyInitialization is still not safe, because another thread could observe a reference to a partially constructedResource.
Listing16.3.UnsafeLazyInitialization.Don'tDothis.
@NotThreadSafe
public class UnsafeLazyInitialization {
private static Resource resource;
public static Resource getInstance() {
if (resource == null)
resource = new Resource(); // unsafe publication
return resource;
}
}
SupposethreadAisthefirsttoinvokegetInstance.Itseesthatresourceisnull,instantiatesanewResource,and setsresourcetoreferenceit.WhenthreadBlatercallsgetInstance,itmightseethatresourcealreadyhasanonͲnull valueandjustusethealreadyconstructedResource.Thismightlookharmlessatfirst,butthereisnohappensͲbefore orderingbetweenthewritingofresourceinAandthereadingofresourceinB.Adataracehasbeenusedtopublishthe object,andthereforeBisnotguaranteedtoseethecorrectstateoftheResource.
TheResourceconstructorchangesthefieldsofthefreshlyallocatedResourcefromtheirdefaultvalues(writtenbythe Objectconstructor)totheirinitialvalues.Sinceneitherthreadusedsynchronization,BcouldpossiblyseeA'sactionsin a different order than A performed them. So even though A initialized the Resource before setting resource to referenceit,BcouldseethewritetoresourceasoccurringbeforethewritestothefieldsoftheResource.Bcouldthus see a partially constructed Resource that may well be in an invalid stateand whose state may unexpectedly change later.
With the exception of immutable objects, it is not safe to use an object that has been initialized by another thread unlessthepublicationhappensͲbeforetheconsumingthreadusesit.
16.2.2.SafePublication
ThesafeͲpublicationidiomsdescribedinChapter3ensurethatthepublishedobjectisvisibletootherthreadsbecause theyensurethepublicationhappensͲbeforetheconsumingthreadloadsareferencetothepublishedobject.IfthreadA placesXonaBlockingQueue(andnothreadsubsequentlymodifiesit)andthreadBretrievesitfromthequeue,Bis guaranteed to see X as A left it. This is because the BlockingQueue implementations have sufficient internal synchronizationtoensurethattheputhappensͲbeforethetake.Similarly,usingasharedvariableguardedbyalockor asharedvolatilevariableensuresthatreadsandwritesofthatvariableareorderedbyhappensͲbefore.
ThishappensͲbeforeguaranteeisactuallyastrongerpromiseofvisibilityandorderingthanmadebysafepublication.
WhenXissafelypublishedfromAtoB,thesafepublicationguaranteesvisibilityofthestateofX,butnotofthestateof othervariablesAmayhavetouched.ButifAputtingXonaqueuehappensͲbeforeBfetchesXfromthatqueue,notonly doesBseeXinthestatethatAleftit(assumingthatXhasnotbeensubsequentlymodifiedbyAoranyoneelse),butB
seeseverythingAdidbeforethehandoff(again,subjecttothesamecaveat).[5]
[5]TheJMMguaranteesthatBseesavalueatleastasupͲtoͲdateasthevaluethatAwrote;subsequentwritesmayormaynotbevisible.
Why did we focus so heavily on @GuardedBy and safe publication, when the JMM already provides us with the more powerful happensͲbefore? Thinking in terms of handing off object ownership and publication fits better into most programdesignsthanthinkingintermsofvisibilityofindividualmemorywrites.ThehappensͲbeforeorderingoperates
7BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel 213
atthelevelofindividualmemoryaccessesͲitisasortof"concurrencyassemblylanguage".Safepublicationoperatesat alevelclosertothatofyourprogram'sdesign.
16.2.3.SafeInitializationIdioms
Itsometimesmakessensetodeferinitializationofobjectsthatareexpensivetoinitializeuntiltheyareactuallyneeded, butwehaveseenhowthemisuseoflazyinitializationcanleadtotrouble.UnsafeLazyInitializationcanbefixedby makingthegeTResourcemethodsynchronized,asshowninListing16.4.BecausethecodepaththroughgetInstance is fairly short (a test and a predicted branch), if getInstance is not called frequently by many threads, there is little enoughcontentionfortheSafeLazyInitializationlockthatthisapproachoffersadequateperformance.
Thetreatmentofstaticfieldswithinitializers(orfieldswhosevalueisinitializedinastaticinitializationblock[JPL2.2.1
and2.5.3])issomewhatspecialandoffersadditionalthreadͲsafetyguarantees.StaticinitializersarerunbytheJVMat classinitializationtime,afterclassloadingbutbeforetheclassisusedbyanythread.BecausetheJVMacquiresalock duringinitialization[JLS12.4.2]andthislockisacquiredbyeachthreadatleastoncetoensurethattheclasshasbeen loaded,memorywritesmadeduringstaticinitializationareautomaticallyvisibletoallthreads.Thusstaticallyinitialized objectsrequirenoexplicitsynchronizationeitherduringconstructionorwhenbeingreferenced.However,thisapplies onlytotheasͲconstructedstateͲiftheobjectismutable,synchronizationisstillrequiredbybothreadersandwritersto makesubsequentmodificationsvisibleandtoavoiddatacorruption.
Listing16.4.ThreadǦsafeLazyInitialization.
@ThreadSafe
public class SafeLazyInitialization {
private static Resource resource;
public synchronized static Resource getInstance() {
if (resource == null)
resource = new Resource();
return resource;
}
}
Listing16.5.EagerInitialization.
@ThreadSafe
public class EagerInitialization {
private static Resource resource = new Resource();
public static Resource getResource() { return resource; }
}
Using eager initialization, shown in Listing 16.5, eliminates the synchronization cost incurred on each call to getInstanceinSafeLazyInitialization.ThistechniquecanbecombinedwiththeJVM'slazyclassloadingtocreate a lazy initialization technique that does not require synchronization on the common code path. The lazy initialization holder class idiom [EJ Item 48] in Listing 16.6 uses a class whose only purpose is to initialize the Resource. The JVM
defersinitializingtheResourceHolderclassuntilitisactuallyused[JLS12.4.1],andbecausetheResourceisinitialized with a static initializer, no additional synchronization is needed. The first call to getresource by any thread causes ResourceHoldertobeloadedandinitialized,atwhichtimetheinitializationoftheResourcehappensthroughthestatic initializer.
Listing16.6.LazyInitializationHolderClassIdiom.
@ThreadSafe
public class ResourceFactory {
private static class ResourceHolder {
public static Resource resource = new Resource();
}
public static Resource getResource() {
return ResourceHolder.resource ;
}
}
16.2.4.DoubleǦcheckedLocking
Nobookonconcurrency wouldbecompletewithoutadiscussionoftheinfamousdoubleͲcheckedlocking(DCL)antiͲ
pattern,showninListing16.7.InveryearlyJVMs,synchronization,evenuncontendedsynchronization,hadasignificant performance cost. As a result, many clever (or at least cleverͲlooking) tricks were invented to reduce the impact of synchronizationͲsomegood,somebad,andsomeugly.DCLfallsintothe"ugly"category.
Again,becausetheperformanceofearlyJVMsleftsomethingtobedesired,lazyinitializationwasoftenusedtoavoid potentially unnecessary expensive operations or reduce application startup time. A properly written lazy initialization
214 JavaConcurrencyInPractice
method requires synchronization. But at the time, synchronization was slow and, more importantly, not completely understood:theexclusionaspectswerewellenoughunderstood,butthevisibilityaspectswerenot.
DCLpurportedtooffer thebestofbothworldsͲlazyinitializationwithoutpayingthesynchronization penaltyonthe commoncodepath.Thewayitworkedwasfirsttocheckwhetherinitializationwasneededwithoutsynchronizing,and iftheresourcereference wasnotnull,useit.Otherwise,synchronizeandcheckagainiftheResourceisinitialized, ensuringthatonlyonethreadactuallyinitializesthesharedResource.ThecommoncodepathͲfetchingareferenceto analreadyconstructedResource doesn'tusesynchronization.Andthat'swheretheproblemis:asdescribedinSection 16.2.1,itispossibleforathreadtoseeapartiallyconstructedResource.
The real problem with DCL is the assumption that the worst thing that can happen when reading a shared object reference without synchronization is to erroneously see a stale value (in this case, null); in that case the DCL idiom compensates for this risk by trying again with the lock held. But the worst case is actually considerably worse Ͳ it is possibletoseeacurrentvalueofthereferencebutstalevaluesfortheobject'sstate,meaningthattheobjectcouldbe seentobeinaninvalidorincorrectstate.
SubsequentchangesintheJMM(Java5.0andlater)haveenabledDCLtoworkifresourceismadevolatile,andthe performanceimpactofthisissmallsincevolatilereadsareusuallyonlyslightlymoreexpensivethannonvolatilereads.
However, this is an idiom whose utility has largely passed Ͳ the forces that motivated it (slow uncontended synchronization, slow JVM startup) are no longer in play, making it less effective as an optimization. The lazy initializationholderidiomoffersthesamebenefitsandiseasiertounderstand.
Listing16.7.DoubleǦcheckedǦlockingAntiǦpattern.
@NotThreadSafe
public class DoubleCheckedLocking {
private static Resource resource;
public static Resource getInstance() {
if (resource == null) {
synchronized (DoubleCheckedLocking.class) {
if (resource == null)
resource = new Resource();
}
}
return resource;
}
}
16.3.InitializationSafety
Theguaranteeofinitializationsafetyallowsproperlyconstructedimmutableobjectstobesafelysharedacrossthreads without synchronization, regardless of how they are publishedeven if published using a data race. (This means that UnsafeLazyInitializationisactuallysafeifResourceisimmutable.)
Without initialization safety, supposedly immutable objects like String can appear to change their value if synchronization is not used by both the publishing and consuming threads. The security architecture relies on the immutabilityofString;thelackofinitializationsafetycouldcreatesecurityvulnerabilitiesthatallowmaliciouscodeto bypasssecuritychecks.
Initializationsafetyguaranteesthatforproperlyconstructedobjects,allthreadswillseethecorrectvaluesoffinalfields thatweresetbytheconstructor,regardlessofhowtheobjectispublished.Further,anyvariablesthatcanbereached throughafinalfieldofaproperlyconstructedobject(suchastheelementsofafinalarrayorthecontentsofaHashMap referencedbyafinalfield)arealsoguaranteedtobevisibletootherthreads.[6]
[6]Thisappliesonlytoobjectsthatarereachableonlythroughfinalfieldsoftheobjectunderconstruction.
Forobjectswithfinalfields,initializationsafetyprohibitsreorderinganypartofconstructionwiththeinitialloadofa referencetothatobject.Allwritestofinalfieldsmadebytheconstructor,aswellastoanyvariablesreachablethrough thosefields,become"frozen"whentheconstructorcompletes,andanythreadthatobtainsareferencetothatobjectis guaranteed to see a value that is at least as up to date as the frozen value. Writes that initialize variables reachable throughfinalfieldsarenotreorderedwithoperationsfollowingthepostͲconstructionfreeze.
7BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel 215
Initialization safety means that SafeStates in Listing 16.8 could be safely published even through unsafe lazy initializationorstashingareferencetoaSafeStatesinapublicstaticfieldwithnosynchronization,eventhoughituses nosynchronizationandreliesonthenonͲthreadͲsafeHashSet.
Listing16.8.InitializationSafetyforImmutableObjects.
@ThreadSafe
public class SafeStates {
private final Map<String, String> states;
public SafeStates() {
states = new HashMap<String, String>();
states.put("alaska", "AK");
states.put("alabama", "AL");
...
states.put("wyoming", "WY");
}
public String getAbbreviation(String s) {
return states.get(s);
}
}
However,anumberofsmallchangestoSafeStateswouldtakeawayitsthreadsafety.Ifstateswerenotfinal,orif anymethodotherthantheconstructormodifieditscontents,initializationsafetywouldnotbestrongenoughtosafely access SafeStates without synchronization. If SafeStates had other nonͲfinal fields, other threads might still see incorrect values of those fields. And allowing the object to escape during construction invalidates the initializationͲ
safetyguarantee.
Initializationsafetymakesvisibilityguaranteesonlyforthevaluesthatarereachablethroughfinalfieldsasofthetime theconstructorfinishes.ForvaluesreachablethroughnonͲfinalfields,orvaluesthatmaychangeafterconstruction,you mustusesynchronizationtoensurevisibility.
Summary
TheJavaMemoryModelspecifieswhentheactionsofonethreadonmemoryareguaranteedtobevisibletoanother.
ThespecificsinvolveensuringthatoperationsareorderedbyapartialorderingcalledhappensͲbefore,whichisspecified at the level of individual memory and synchronization operations. In the absence of sufficient synchronization, some verystrangethingscanhappenwhenthreadsaccessshareddata.However,thehigherͲlevelrulesofferedinChapters2
and3,suchas@GuardedByandsafepublication,canbeusedtoensurethreadsafetywithoutresortingtothelowͲlevel detailsofhappensͲbefore.
216 JavaConcurrencyInPractice
AppendixA.AnnotationsforConcurrency
We've used annotations such as @GuardedBy and @ThreadSafe to show how threadͲsafety promises and synchronization policies can be documented. This appendix documents these annotations; their source code can be downloaded from this book's website. (There are, of course, additional threadͲsafety promises and implementation detailsthatshouldbedocumentedbutthatarenotcapturedbythisminimalsetofannotations.) A.1.ClassAnnotations
WeusethreeclassͲlevelannotationstodescribeaclass'sintendedthreadͲsafetypromises:@Immutable,@ThreadSafe, and @NotThreadSafe. @Immutable means, of course, that the class is immutable, and implies @ThreadSafe.
@NotThreadSafeisoptionalͲifaclassisnotannotatedasthreadͲsafe,itshouldbepresumednottobethreadͲsafe,but ifyouwanttomakeitextraclear,use@NotThreadSafe.
These annotations are relatively unintrusive and are beneficial to both users and maintainers. Users can see immediately whether a class is threadͲsafe, and maintainers can see immediately whether threadͲsafety guarantees mustbepreserved.Annotationsarealsousefultoathirdconstituency:tools.Staticcodeanalysistoolsmaybeableto verifythatthecodecomplieswiththecontractindicatedbytheannotation,suchasverifyingthataclassannotatedwith
@Immutableactuallyisimmutable.
A.2.FieldandMethodAnnotations
TheclassͲlevelannotationsabovearepartofthepublicdocumentationfortheclass.Otheraspectsofaclass'sthreadͲ
safetystrategyareentirelyformaintainersandarenotpartofitspublicdocumentation.
Classesthatuselockingshoulddocumentwhichstatevariablesareguardedwithwhichlocks,andwhichlocksareused to guard those variables. A common source of inadvertent nonͲthreadͲsafety is when a threadͲsafe class consistently useslockingtoguarditsstate,butislatermodifiedtoaddeithernewstatevariablesthatarenotadequatelyguardedby locking, or new methods that do not use locking properly to guard the existing state variables. Documenting which variablesareguardedbywhichlockscanhelppreventbothtypesofomissions.
@GuardedBy(lock) documents that a field or method should be accessed only with a specific lock held. The lock argumentidentifiesthelockthatshouldbeheldwhenaccessingtheannotatedfieldormethod.Thepossiblevaluesfor lockare:
x @GuardedBy("this"),meaningtheintrinsiclockonthecontainingobject(theobjectofwhichthemethodor fieldisamember);
x @GuardedBy("fieldName"),meaningthelockassociatedwiththeobjectreferencedbythenamedfield,either anintrinsiclock(forfieldsthatdonotrefertoaLock)oranexplicitLock(forfieldsthatrefertoaLock); x @GuardedBy("ClassName.fieldName"),like@GuardedBy("fieldName"),butreferencingalockobjectheldina staticfieldofanotherclass;
x @GuardedBy("methodName()"),meaningthelockobjectthatisreturnedbycallingthenamedmethod; x @GuardedBy("ClassName.class"),meaningtheclassliteralobjectforthenamedclass.
Using@GuardedBytoidentifyeachstatevariablethatneedslockingandwhichlockguardsitcanassistinmaintenance andcodereviews,andcanhelpautomatedanalysistoolsspotpotentialthreadͲsafetyerrors.
7BPartIV:AdvancedTopicsͲ30BBibliography
217
Bibliography
KenArnold,JamesGosling,andDavidHolmes.
David F. Bacon, Ravi B. Konuru, Chet Murthy, and Mauricio J. Serrano.
URLhttp://citeseer.ist.psu.edu/bacon98thin.html.
JoshuaBloch.
JoshuaBlochandNealGafter.
Hans Boehm.
URLhttp://doi.acm.org/10.1145/604131.604153.
HansBoehm.
URLhttp://developers.sun.com/learning/javaoneonline/2005/coreplatform/TSͲ3281.pdf.
JosephBowbeer.
URLhttp://java.sun.com/products/jfc/tsc/articles/threads/threads3.html.
CliffClick.
CliffClick.
URLhttp://developers.sun.com/learning/javaoneonline/2005/coreplatform/TSͲ3268.pdf.
MartinFowler.
ErichGamma,RichardHelm,RalphJohnson,andJohnVlissides.
Martin Gardner.
JamesGosling,BillJoy,GuySteele,andGiladBracha.
Tim Harris and Keir Fraser. Language Support for Lightweight Transactions. In OOPSLA '03: Proceedings of the 18th Annual ACM SIGPLAN Conference on ObjectͲOriented Programming, Systems, Languages, and Applications, pages 388402.ACMPress,2003.URLhttp://doi.acm.org/10.1145/949305.949340.
TimHarris,SimonMarlow,SimonPeytonͲJones,andMauriceHerlihy.
ACMPress,2005.URLhttp://doi.acm.org/10.1145/1065944.1065952.
Maurice Herlihy. WaitͲFree Synchronization.
149,1991.URLhttp://doi.acm.org/10.1145/114005.102808.
Maurice Herlihy and Nir Shavit.
C.A.R.Hoare.
URLhttp://doi.acm.org/10.1145/355620.361161.
DavidHovemeyerandWilliamPugh.
URLhttp://doi.acm.org/10.1145/1052883.1052895.
218 JavaConcurrencyInPractice
RamnivasLaddad.
DougLea.
DougLea.
J.D.C.Little.
JeremyManson,WilliamPugh,andSaritaV.Adve.
URLhttp://doi.acm.org/10.1145/1040305.1040336.
GeorgeMarsaglia.
Maged M. Michael and Michael L. Scott.
URLhttp://citeseer.ist.psu.edu/michael96simple.html.
MarkMoirandNirShavit.
Press,2004.
WilliamPughandJeremyManson.
URLhttp://www.cs.umd.edu/~pugh/java/memoryModel/jsr133.pdf.
M.Raynal.
WilliamN.Scherer,DougLea,andMichaelL.Scott.
R.K.Treiber.
AndrewWellings.
Document Outline
Java Concurrency
In Practice
Index
Listing and Image Index
Preface
How to Use this Book
Code Examples
Listing 1. Bad Way to Sort a List. Don't Do this.
Listing 2. Less than Optimal Way to Sort a List.
Acknowledgments
Chapter 1 - Introduction
1.1. A (Very) Brief History of Concurrency
1.2. Benefits of Threads
1.2.1. Exploiting Multiple Processors
1.2.2. Simplicity of Modeling
1.2.3. Simplified Handling of Asynchronous Events
1.2.4. More Responsive User Interfaces
1.3. Risks of Threads
1.3.1. Safety Hazards
Listing 1.1. Non-thread-safe Sequence Generator.
Figure 1.1. Unlucky Execution of UnsafeSequence.Nextvalue.
Listing 1.2. Thread-safe Sequence Generator.
1.3.2. Liveness Hazards
1.3.3. Performance Hazards
1.4. Threads are Everywhere
Part I: Fundamentals
Chapter 2. Thread Safety
2.1. What is Thread Safety?
2.1.1. Example: A Stateless Servlet
Listing 2.1. A Stateless Servlet.
2.2. Atomicity
Listing 2.2. Servlet that Counts Requests without the Necessary Synchronization. Don't Do this.
2.2.1. Race Conditions
2.2.2. Example: Race Conditions in Lazy Initialization
Listing 2.3. Race Condition in Lazy Initialization. Don't Do this.
2.2.3. Compound Actions
Listing 2.4. Servlet that Counts Requests Using AtomicLong.
2.3. Locking
Listing 2.5. Servlet that Attempts to Cache its Last Result without Adequate Atomicity. Don't Do this.
2.3.1. Intrinsic Locks
Listing 2.6. Servlet that Caches Last Result, But with Unacceptably Poor Concurrency. Don't Do this.
2.3.2. Reentrancy
Listing 2.7. Code that would Deadlock if Intrinsic Locks were Not Reentrant.
2.4. Guarding State with Locks
2.5. Liveness and Performance
Figure 2.1. Poor Concurrency of SynchronizedFactorizer.
Listing 2.8. Servlet that Caches its Last Request and Result.
Chapter 3. Sharing Objects
3.1. Visibility
Listing 3.1. Sharing Variables without Synchronization. Don't Do this.
3.1.1. Stale Data
Listing 3.2. Non-thread-safe Mutable Integer Holder.
Listing 3.3. Thread-safe Mutable Integer Holder.
3.1.2. Non-atomic 64-bit Operations
3.1.3. Locking and Visibility
Figure 3.1. Visibility Guarantees for Synchronization.
3.1.4. Volatile Variables
Listing 3.4. Counting Sheep.
3.2. Publication and Escape
Listing 3.5. Publishing an Object.
Listing 3.6. Allowing Internal Mutable State to Escape. Don't Do this.
Listing 3.7. Implicitly Allowing the this Reference to Escape. Don't Do this.
3.2.1. Safe Construction Practices
Listing 3.8. Using a Factory Method to Prevent the this Reference from Escaping During Construction.
3.3. Thread Confinement
3.3.1. Ad-hoc Thread Confinement
3.3.2. Stack Confinement
Listing 3.9. Thread Confinement of Local Primitive and Reference Variables.
3.3.3. ThreadLocal
Listing 3.10. Using ThreadLocal to Ensure thread Confinement.
3.4. Immutability
Listing 3.11. Immutable Class Built Out of Mutable Underlying Objects.
3.4.1. Final Fields
3.4.2. Example: Using Volatile to Publish Immutable Objects
Listing 3.12. Immutable Holder for Caching a Number and its Factors.
3.5. Safe Publication
Listing 3.13. Caching the Last Result Using a Volatile Reference to an Immutable Holder Object.
Listing 3.14. Publishing an Object without Adequate Synchronization. Don't Do this.
3.5.1. Improper Publication: When Good Objects Go Bad
Listing 3.15. Class at Risk of Failure if Not Properly Published.
3.5.2. Immutable Objects and Initialization Safety
3.5.3. Safe Publication Idioms
3.5.4. Effectively Immutable Objects
3.5.5. Mutable Objects
3.5.6. Sharing Objects Safely
Chapter 4. Composing Objects
4.1. Designing a Thread-safe Class
Listing 4.1. Simple Thread-safe Counter Using the Java Monitor Pattern.
4.1.1. Gathering Synchronization Requirements
4.1.2. State-dependent Operations
4.1.3. State Ownership
4.2. Instance Confinement
Listing 4.2. Using Confinement to Ensure Thread Safety.
4.2.1. The Java Monitor Pattern
Listing 4.3. Guarding State with a Private Lock.
4.2.2. Example: Tracking Fleet Vehicles
4.3. Delegating Thread Safety
Listing 4.4. Monitor-based Vehicle Tracker Implementation.
Listing 4.5. Mutable Point Class Similar to Java.awt.Point.
4.3.1. Example: Vehicle Tracker Using Delegation
Listing 4.6. Immutable Point class used by DelegatingVehicleTracker.
Listing 4.7. Delegating Thread Safety to a ConcurrentHashMap.
Listing 4.8. Returning a Static Copy of the Location Set Instead of a "Live" One.
4.3.2. Independent State Variables
Listing 4.9. Delegating Thread Safety to Multiple Underlying State Variables.
4.3.3. When Delegation Fails
Listing 4.10. Number Range Class that does Not Sufficiently Protect Its Invariants. Don't Do this.
4.3.4. Publishing Underlying State Variables
4.3.5. Example: Vehicle Tracker that Publishes Its State
Listing 4.11. Thread-safe Mutable Point Class.
Listing 4.12. Vehicle Tracker that Safely Publishes Underlying State.
4.4. Adding Functionality to Existing Thread-safe Classes
Listing 4.13. Extending Vector to have a Put-if-absent Method.
4.4.1. Client-side Locking
Listing 4.14. Non-thread-safe Attempt to Implement Put-if-absent. Don't Do this.
Listing 4.15. Implementing Put-if-absent with Client-side Locking.
4.4.2. Composition
Listing 4.16. Implementing Put-if-absent Using Composition.
4.5. Documenting Synchronization Policies
4.5.1. Interpreting Vague Documentation
Chapter 5. Building Blocks
5.1. Synchronized Collections
5.1.1. Problems with Synchronized Collections
Listing 5.1. Compound Actions on a Vector that may Produce Confusing Results.
Figure 5.1. Interleaving of Getlast and Deletelast that throws ArrayIndexOutOfBoundsException.
Listing 5.2. Compound Actions on Vector Using Client-side Locking.
Listing 5.3. Iteration that may Throw ArrayIndexOutOfBoundsException.
Listing 5.4. Iteration with Client-side Locking.
5.1.2. Iterators and Concurrentmodificationexception
Listing 5.5. Iterating a List with an Iterator.
5.1.3. Hidden Iterators
Listing 5.6. Iteration Hidden within String Concatenation. Don't Do this.
5.2. Concurrent Collections
5.2.1. ConcurrentHashMap
5.2.2. Additional Atomic Map Operations
5.2.3. CopyOnWriteArrayList
Listing 5.7. ConcurrentMap Interface.
5.3. Blocking Queues and the Producer-consumer Pattern
5.3.1. Example: Desktop Search
5.3.2. Serial Thread Confinement
Listing 5.8. Producer and Consumer Tasks in a Desktop Search Application.
Listing 5.9. Starting the Desktop Search.
5.3.3. Deques and Work Stealing
5.4. Blocking and Interruptible Methods
Listing 5.10. Restoring the Interrupted Status so as Not to Swallow the Interrupt.
5.5. Synchronizers
5.5.1. Latches
5.5.2. FutureTask
Listing 5.11. Using CountDownLatch for Starting and Stopping Threads in Timing Tests.
Listing 5.12. Using FutureTask to Preload Data that is Needed Later.
Listing 5.13. Coercing an Unchecked Throwable to a RuntimeException.
5.5.3. Semaphores
5.5.4. Barriers
Listing 5.14. Using Semaphore to Bound a Collection.
5.6. Building an Efficient, Scalable Result Cache
Listing 5.16. Initial Cache Attempt Using HashMap and Synchronization.
Listing 5.17. Replacing HashMap with ConcurrentHashMap.
Figure 5.4. Unlucky Timing that could Cause Memorizer3 to Calculate the Same Value Twice.
Listing 5.18. Memorizing Wrapper Using FutureTask.
Listing 5.19. Final Implementation of Memorizer.
Listing 5.20. Factorizing Servlet that Caches Results Using Memorizer.
Summary of Part I
Part II: Structuring Concurrent Applications
Chapter 6. Task Execution
6.1. Executing Tasks in Threads
6.1.1. Executing Tasks Sequentially
Listing 6.1. Sequential Web Server.
6.1.2. Explicitly Creating Threads for Tasks
Listing 6.2. Web Server that Starts a New Thread for Each Request.
6.1.3. Disadvantages of Unbounded Thread Creation
6.2. The Executor Framework
Listing 6.3. Executor Interface.
6.2.1. Example: Web Server Using Executor
Listing 6.4. Web Server Using a Thread Pool.
Listing 6.5. Executor that Starts a New Thread for Each Task.
6.2.2. Execution Policies
Listing 6.6. Executor that Executes Tasks Synchronously in the Calling Thread.
6.2.3. Thread Pools
6.2.4. Executor Lifecycle
Listing 6.7. Lifecycle Methods in ExecutorService.
Listing 6.8. Web Server with Shutdown Support.
6.2.5. Delayed and Periodic Tasks
6.3. Finding Exploitable Parallelism
Listing 6.9. Class Illustrating Confusing Timer Behavior.
6.3.1. Example: Sequential Page Renderer
Listing 6.10. Rendering Page Elements Sequentially.
6.3.2. Result-bearing Tasks: Callable and Future
Listing 6.11. Callable and Future Interfaces.
Listing 6.12. Default Implementation of newTaskFor in ThreadPoolExecutor.
6.3.3. Example: Page Renderer with Future
6.3.4. Limitations of Parallelizing Heterogeneous Tasks
Listing 6.13. Waiting for Image Download with Future.
6.3.5. CompletionService: Executor Meets BlockingQueue
Listing 6.14. QueueingFuture Class Used By ExecutorCompletionService.
6.3.6. Example: Page Renderer with CompletionService
Listing 6.15. Using CompletionService to Render Page Elements as they Become Available.
6.3.7. Placing Time Limits on Tasks
6.3.8. Example: A Travel Reservations Portal
Listing 6.16. Fetching an Advertisement with a Time Budget.
Summary
Listing 6.17. Requesting Travel Quotes Under a Time Budget.
Chapter 7. Cancellation and Shutdown
7.1. Task Cancellation
Listing 7.1. Using a Volatile Field to Hold Cancellation State.
Listing 7.2. Generating a Second's Worth of Prime Numbers.
7.1.1. Interruption
Listing 7.3. Unreliable Cancellation that can Leave Producers Stuck in a Blocking Operation. Don't Do this.
Listing 7.4. Interruption Methods in Thread.
Listing 7.5. Using Interruption for Cancellation.
7.1.2. Interruption Policies
7.1.3. Responding to Interruption
Listing 7.6. Propagating InterruptedException to Callers.
Listing 7.7. Non-cancelable Task that Restores Interruption Before Exit.
7.1.4. Example: Timed Run
Listing 7.8. Scheduling an Interrupt on a Borrowed Thread. Don't Do this.
7.1.5. Cancellation Via Future
Listing 7.9. Interrupting a Task in a Dedicated Thread.
Listing 7.10. Cancelling a Task Using Future.
7.1.7. Encapsulating Nonstandard Cancellation with Newtaskfor
Listing 7.11. Encapsulating Nonstandard Cancellation in a Thread by Overriding Interrupt.
7.2. Stopping a Thread-based Service
7.2.1. Example: A Logging Service
Listing 7.12. Encapsulating Nonstandard Cancellation in a Task with Newtaskfor.
Listing 7.13. Producer-Consumer Logging Service with No Shutdown Support.
Listing 7.14. Unreliable Way to Add Shutdown Support to the Logging Service.
7.2.2. ExecutorService Shutdown
Listing 7.15. Adding Reliable Cancellation to LogWriter.
Listing 7.16. Logging Service that Uses an ExecutorService.
7.2.3. Poison Pills
Listing 7.17. Shutdown with Poison Pill.
7.2.4. Example: A One-shot Execution Service
Listing 7.18. Producer Thread for IndexingService.
Listing 7.19. Consumer Thread for IndexingService.
Listing 7.20. Using a Private Executor Whose Lifetime is Bounded by a Method Call.
7.2.5. Limitations of Shutdownnow
Listing 7.21. ExecutorService that Keeps Track of Cancelled Tasks After Shutdown.
Listing 7.22. Using TRackingExecutorService to Save Unfinished Tasks for Later Execution.
7.3. Handling Abnormal Thread Termination
Listing 7.23. Typical Thread-pool Worker Thread Structure.
7.3.1. Uncaught Exception Handlers
Listing 7.24. UncaughtExceptionHandler Interface.
Listing 7.25. UncaughtExceptionHandler that Logs the Exception.
7.4. JVM Shutdown
7.4.1. Shutdown Hooks
Listing 7.26. Registering a Shutdown Hook to Stop the Logging Service.
7.4.2. Daemon Threads
7.4.3. Finalizers
Summary
Chapter 8. Applying Thread Pools
8.1. Implicit Couplings Between Tasks and Execution Policies
8.1.1. Thread Starvation Deadlock
Listing 8.1. Task that Deadlocks in a Single-threaded Executor. Don't Do this.
8.1.2. Long-running Tasks
8.2. Sizing Thread Pools
8.3. Configuring ThreadPoolExecutor
8.3.1. Thread Creation and Teardown
Listing 8.2. General Constructor for ThreadPoolExecutor.
8.3.2. Managing Queued Tasks
8.3.3. Saturation Policies
Listing 8.3. Creating a Fixed-sized Thread Pool with a Bounded Queue and the Caller-runs Saturation Policy.
8.3.4. Thread Factories
Listing 8.4. Using a Semaphore to Throttle Task Submission.
Listing 8.5. ThreadFactory Interface.
Listing 8.6. Custom Thread Factory.
8.3.5. Customizing ThreadPoolExecutor After Construction
Listing 8.7. Custom Thread Base Class.
Listing 8.8. Modifying an Executor Created with the Standard Factories.
8.4. Extending ThreadPoolExecutor
8.4.1. Example: Adding Statistics to a Thread Pool
Listing 8.9. Thread Pool Extended with Logging and Timing.
8.5. Parallelizing Recursive Algorithms
Listing 8.10. Transforming Sequential Execution into Parallel Execution.
Listing 8.11. Transforming Sequential Tail-recursion into Parallelized Recursion.
Listing 8.12. Waiting for Results to be Calculated in Parallel.
8.5.1. Example: A Puzzle Framework
Listing 8.13. Abstraction for Puzzles Like the "Sliding Blocks Puzzle".
Listing 8.14. Link Node for the Puzzle Solver Framework.
Listing 8.15. Sequential Puzzle Solver.
Listing 8.16. Concurrent Version of Puzzle Solver.
Listing 8.17. Result-bearing Latch Used by ConcurrentPuzzleSolver.
Listing 8.18. Solver that Recognizes when No Solution Exists.
Summary
Chapter 9. GUI Applications
9.1. Why are GUIs Single-threaded?
9.1.1. Sequential Event Processing
9.1.2. Thread Confinement in Swing
9.2. Short-running GUI Tasks
Figure 9.1. Control Flow of a Simple Button Click.
Listing 9.1. Implementing SwingUtilities Using an Executor.
Listing 9.2. Executor Built Atop SwingUtilities.
Listing 9.3. Simple Event Listener.
Figure 9.2. Control Flow with Separate Model and View Objects.
9.3. Long-running GUI Tasks
Listing 9.4. Binding a Long-running Task to a Visual Component.
Listing 9.5. Long-running Task with User Feedback.
9.3.1. Cancellation
Listing 9.6. Cancelling a Long-running Task.
9.3.2. Progress and Completion Indication
9.3.3. SwingWorker
9.4. Shared Data Models
Listing 9.7. Background Task Class Supporting Cancellation, Completion Notification, and Progress Notification.
Listing 9.8. Initiating a Long-running, Cancellable Task with BackgroundTask.
9.4.1. Thread-safe Data Models
9.4.2. Split Data Models
9.5. Other Forms of Single-threaded Subsystems
Summary
Part III: Liveness, Performance, and Testing
Chapter 10. Avoiding Liveness Hazards
10.1. Deadlock
10.1.1. Lock-ordering Deadlocks
Figure 10.1. Unlucky Timing in LeftRightDeadlock.
Listing 10.1. Simple Lock-ordering Deadlock. Don't Do this.
10.1.2. Dynamic Lock Order Deadlocks
Listing 10.2. Dynamic Lock-ordering Deadlock. Don't Do this.
Listing 10.3. Inducing a Lock Ordering to Avoid Deadlock.
Listing 10.4. Driver Loop that Induces Deadlock Under Typical Conditions.
10.1.3. Deadlocks Between Cooperating Objects
10.1.4. Open Calls
Listing 10.5. Lock-ordering Deadlock Between Cooperating Objects. Don't Do this.
10.1.5. Resource Deadlocks
Listing 10.6. Using Open Calls to Avoiding Deadlock Between Cooperating Objects.
10.2. Avoiding and Diagnosing Deadlocks
10.2.1. Timed Lock Attempts
10.2.2. Deadlock Analysis with Thread Dumps
Listing 10.7. Portion of Thread Dump After Deadlock.
10.3. Other Liveness Hazards
10.3.1. Starvation
10.3.2. Poor Responsiveness
10.3.3. Livelock
Summary
Chapter 11. Performance and Scalability
11.1. Thinking about Performance
11.1.1. Performance Versus Scalability
11.1.2. Evaluating Performance Tradeoffs
11.2. Amdahl's Law
Figure 11.1. Maximum Utilization Under Amdahl's Law for Various Serialization Percentages.
Listing 11.1. Serialized Access to a Task Queue.
11.2.1. Example: Serialization Hidden in Frameworks
Figure 11.2. Comparing Queue Implementations.
11.2.2. Applying Amdahl's Law Qualitatively
11.3. Costs Introduced by Threads
11.3.1. Context Switching
Listing 11.2. Synchronization that has No Effect. Don't Do this.
11.3.2. Memory Synchronization
Listing 11.3. Candidate for Lock Elision.
11.3.3. Blocking
11.4. Reducing Lock Contention
11.4.1. Narrowing Lock Scope ("Get in, Get Out")
Listing 11.4. Holding a Lock Longer than Necessary.
Listing 11.5. Reducing Lock Duration.
11.4.2. Reducing Lock Granularity
Listing 11.6. Candidate for Lock Splitting.
Listing 11.7. ServerStatus Refactored to Use Split Locks.
11.4.3. Lock Striping
11.4.4. Avoiding Hot Fields
Listing 11.8. Hash-based Map Using Lock Striping.
11.4.5. Alternatives to Exclusive Locks
11.4.6. Monitoring CPU Utilization
11.4.7. Just Say No to Object Pooling
11.5. Example: Comparing Map Performance
Figure 11.3. Comparing Scalability of Map Implementations.
11.6. Reducing Context Switch Overhead
Summary
Chapter 12. Testing Concurrent Programs
12.1. Testing for Correctness
Listing 12.1. Bounded Buffer Using Semaphore.
12.1.1. Basic Unit Tests
Listing 12.2. Basic Unit Tests for BoundedBuffer.
12.1.2. Testing Blocking Operations
Listing 12.3. Testing Blocking and Responsiveness to Interruption.
12.1.3. Testing Safety
Listing 12.4. Medium-quality Random Number Generator Suitable for Testing.
Listing 12.5. Producer-consumer Test Program for BoundedBuffer.
Listing 12.6. Producer and Consumer Classes Used in PutTakeTest.
12.1.4. Testing Resource Management
12.1.5. Using Callbacks
Listing 12.7. Testing for Resource Leaks.
Listing 12.8. Thread Factory for Testing ThreadPoolExecutor.
Listing 12.9. Test Method to Verify Thread Pool Expansion.
12.1.6. Generating More Interleavings
Listing 12.10. Using Thread.yield to Generate More Interleavings.
12.2. Testing for Performance
12.2.1. Extending PutTakeTest to Add Timing
Listing 12.11. Barrier-based Timer.
Figure 12.1. TimedPutTakeTest with Various Buffer Capacities.
Listing 12.12. Testing with a Barrier-based Timer.
Listing 12.13. Driver Program - for TimedPutTakeTest.
12.2.2. Comparing Multiple Algorithms
Figure 12.2. Comparing Blocking Queue Implementations.
12.2.3. Measuring Responsiveness
Figure 12.3. Completion Time Histogram for TimedPutTakeTest with Default (Non-fair) and Fair Semaphores.
Figure 12.4. Completion Time Histogram for TimedPutTakeTest with Single-item Buffers.
12.3. Avoiding Performance Testing Pitfalls
12.3.1. Garbage Collection
12.3.2. Dynamic Compilation
Figure 12.5. Results Biased by Dynamic Compilation.
12.3.3. Unrealistic Sampling of Code Paths
12.3.4. Unrealistic Degrees of Contention
12.3.5. Dead Code Elimination
12.4. Complementary Testing Approaches
12.4.1. Code Review
12.4.2. Static Analysis Tools
12.4.3. Aspect-oriented Testing Techniques
12.4.4. Profilers and Monitoring Tools
Summary
Part IV: Advanced Topics
Chapter 13 - Explicit Locks
13.1. Lock and ReentrantLock
Listing 13.1. Lock Interface.
Listing 13.2. Guarding Object State Using ReentrantLock.
13.1.1. Polled and Timed Lock Acquisition
13.1.2. Interruptible Lock Acquisition
Listing 13.3. Avoiding Lock-ordering Deadlock Using trylock.
Listing 13.4. Locking with a Time Budget.
Listing 13.5. Interruptible Lock Acquisition.
13.1.3. Non-block-structured Locking
13.2. Performance Considerations
Figure 13.1. Intrinsic Locking Versus ReentrantLock Performance on Java 5.0 and Java 6.
13.3. Fairness
Figure 13.2. Fair Versus Non-fair Lock Performance.
13.4. Choosing Between Synchronized and ReentrantLock
13.5. Read-write Locks
Listing 13.6. ReadWriteLock Interface.
Listing 13.7. Wrapping a Map with a Read-write Lock.
Figure 13.3. Read-write Lock Performance.
Summary
Chapter 14 - Building Custom Synchronizers
14.1. Managing State Dependence
Listing 14.1. Structure of Blocking State-dependent Actions.
14.1.1. Example: Propagating Precondition Failure to Callers
Listing 14.2. Base Class for Bounded Buffer Implementations.
Listing 14.3. Bounded Buffer that Balks When Preconditions are Not Met.
Listing 14.4. Client Logic for Calling GrumpyBoundedBuffer.
14.1.2. Example: Crude Blocking by Polling and Sleeping
Figure 14.1. Thread Oversleeping Because the Condition Became True Just After It Went to Sleep.
Listing 14.5. Bounded Buffer Using Crude Blocking.
14.1.3. Condition Queues to the Rescue
Listing 14.6. Bounded Buffer Using Condition Queues.
14.2. Using Condition Queues
14.2.1. The Condition Predicate
14.2.2. Waking Up Too Soon
Listing 14.7. Canonical Form for State-dependent Methods.
14.2.3. Missed Signals
14.2.4. Notification
Listing 14.8. Using Conditional Notification in BoundedBuffer.put.
14.2.5. Example: A Gate Class
14.2.6. Subclass Safety Issues
Listing 14.9. Recloseable Gate Using Wait and Notifyall.
14.2.7. Encapsulating Condition Queues
14.2.8. Entry and Exit Protocols
14.3. Explicit Condition Objects
Listing 14.10. Condition Interface.
14.4. Anatomy of a Synchronizer
Listing 14.11. Bounded Buffer Using Explicit Condition Variables.
Listing 14.12. Counting Semaphore Implemented Using Lock.
14.5. AbstractQueuedSynchronizer
Listing 14.13. Canonical Forms for Acquisition and Release in AQS.
14.5.1. A Simple Latch
Listing 14.14. Binary Latch Using AbstractQueuedSynchronizer.
14.6. AQS in Java.util.concurrent Synchronizer Classes
14.6.1. ReentrantLock
Listing 14.15. tryAcquire Implementation From Non-fair ReentrantLock.
14.6.2. Semaphore and CountDownLatch
Listing 14.16. tryacquireshared and tryreleaseshared from Semaphore.
14.6.3. FutureTask
14.6.4. ReentrantReadWriteLock
Summary
Chapter 15. Atomic Variables and Non-blocking Synchronization
15.1. Disadvantages of Locking
15.2. Hardware Support for Concurrency
15.2.1. Compare and Swap
Listing 15.1. Simulated CAS Operation.
15.2.2. A Non-blocking Counter
Listing 15.2. Non-blocking Counter Using CAS.
15.2.3. CAS Support in the JVM
15.3. Atomic Variable Classes
15.3.1. Atomics as "Better Volatiles"
Listing 15.3. Preserving Multivariable Invariants Using CAS.
15.3.2. Performance Comparison: Locks Versus Atomic Variables
Figure 15.1. Lock and AtomicInteger Performance Under High Contention.
Figure 15.2. Lock and AtomicInteger Performance Under Moderate Contention.
Listing 15.4. Random Number Generator Using ReentrantLock.
Listing 15.5. Random Number Generator Using AtomicInteger.
15.4. Non-blocking Algorithms
15.4.1. A Non-blocking Stack
15.4.2. A Non-blocking Linked List
Listing 15.6. Non-blocking Stack Using Treiber's Algorithm (Treiber, 1986).
Figure 15.3. Queue with Two Elements in Quiescent State.
Figure 15.4. Queue in Intermediate State During Insertion.
Figure 15.5. Queue Again in Quiescent State After Insertion is Complete.
Listing 15.7. Insertion in the Michael-Scott Non-blocking Queue Algorithm (Michael and Scott, 1996).
15.4.3. Atomic Field Updaters
Listing 15.8. Using Atomic Field Updaters in ConcurrentLinkedQueue.
15.4.4. The ABA Problem
Summary
Chapter 16. The Java Memory Model
16.1.1. Platform Memory Models
16.1.2. Reordering
Figure 16.1. Interleaving Showing Reordering in PossibleReordering.
16.1.3. The Java Memory Model in 500 Words or Less
Listing 16.1. Insufficiently Synchronized Program that can have Surprising Results. Don't Do this.
Figure 16.2. Illustration of Happens-before in the Java Memory Model.
16.1.4. Piggybacking on Synchronization
Listing 16.2. Inner Class of FutureTask Illustrating Synchronization Piggybacking.
16.2. Publication
16.2.1. Unsafe Publication
Listing 16.3. Unsafe Lazy Initialization. Don't Do this.
16.2.2. Safe Publication
16.2.3. Safe Initialization Idioms
Listing 16.4. Thread-safe Lazy Initialization.
Listing 16.5. Eager Initialization.
Listing 16.6. Lazy Initialization Holder Class Idiom.
16.2.4. Double-checked Locking
Listing 16.7. Double-checked-locking Anti-pattern. Don't Do this.
Listing 16.8. Initialization Safety for Immutable Objects.
Summary
Appendix A. Annotations for Concurrency
A.1. Class Annotations
A.2. Field and Method Annotations
Bibliography
Word Bookmarks
pref03lev1sec1
pref03lev1sec2
pref03fn01
pref03list01
pref03list02
pref03lev1sec3
iddle1387
iddle1388
iddle1457
iddle2321
iddle2349
iddle3371
iddle3372
iddle3373
iddle3674
iddle3675
iddle3941
iddle4147
iddle4231
iddle4232
iddle4280
iddle4281
iddle4323
iddle4324
iddle1133
iddle1634
iddle1756
iddle2033
iddle2984
iddle3228
iddle3229
iddle3676
iddle3677
iddle3696
iddle3697
iddle3698
iddle4159
iddle4168
iddle4228
iddle4294
iddle4295
iddle4741
iddle4742
iddle4743
iddle4810
iddle4935
iddle4936
iddle3232
iddle3233
iddle3265
iddle4099
iddle4284
iddle4285
iddle4746
iddle4879
ch01lev2sec1
ch01lev2sec2
iddle1130
iddle1131
iddle2066
iddle2413
iddle2683
iddle3340
iddle3341
iddle4004
iddle4171
iddle4203
ch01lev2sec3
ch01fn01
ch01lev2sec4
iddle1979
iddle1980
iddle2061
iddle2062
iddle2068
iddle2069
iddle2497
iddle2765
iddle2766
iddle2767
iddle2776
iddle2777
iddle3176
iddle3177
iddle4573
iddle4740
iddle4777
iddle4778
iddle4813
iddle5028
iddle5029
ch01lev2sec5
ch01list01
iddle1094
iddle1635
iddle1636
iddle1637
iddle1959
iddle2537
iddle2768
iddle2769
iddle3338
iddle4465
iddle4466
ch01fig01
ch01sb01
ch01fn02
iddle1395
iddle1396
iddle2175
iddle2481
iddle2621
iddle2622
iddle2703
iddle3415
iddle3416
iddle3544
iddle3545
iddle3776
iddle3777
iddle3778
iddle4020
iddle4021
iddle4022
iddle4566
iddle4860
iddle48911
iddle4915
ch01fn03
ch01list02
iddle1261
iddle1279
iddle1614
iddle1693
iddle1694
iddle1709
iddle1758
iddle1797
iddle1991
ch01lev2sec6
iddle2074
iddle2075
iddle2616
iddle2950
iddle3014
iddle3015
iddle3016
iddle3028
iddle3031
iddle3037
iddle3038
iddle3422
iddle3423
iddle3424
iddle3425
iddle3426
iddle3477
iddle3478
iddle3479
iddle3480
iddle3481
iddle3494
iddle3495
iddle3511
iddle3512
iddle3543
iddle3907
iddle3908
iddle3974
iddle4087
iddle4227
iddle4372
iddle4759
iddle4760
iddle4761
iddle4867
iddle4878
ch01lev2sec7
iddle1180
iddle2411
iddle2412
iddle2928
iddle4000
iddle4001
iddle4383
iddle4384
iddle4527
iddle4822
iddle4934
ch01sb02
iddle1107
iddle1108
iddle1111
iddle1181
iddle1583
iddle1584
iddle2487
iddle2912
iddle2913
iddle3843
iddle3844
iddle4002
iddle4003
iddle4103
iddle4104
iddle4200
iddle4201
iddle4202
iddle4210
iddle4211
iddle4414
iddle4528
ch01fn04
iddle1017
iddle1018
iddle1019
iddle1254
iddle1451
iddle1452
iddle1453
iddle1531
iddle1532
iddle1639
iddle1640
iddle1652
iddle1746
iddle1998
iddle1999
iddle2000
iddle2001
iddle2002
iddle2003
iddle2347
iddle2348
iddle2870
iddle2871
iddle2872
iddle3027
iddle3032
iddle3033
iddle3034
iddle3262
iddle3266
iddle3267
iddle3268
iddle3278
iddle3281
iddle3282
iddle3448
iddle3449
iddle4028
iddle4029
iddle4030
iddle4031
iddle4158
iddle4222
iddle4223
iddle4239
iddle4240
iddle4377
iddle4378
iddle4379
iddle4380
iddle4381
iddle4382
iddle4406
iddle4407
iddle4532
iddle4533
iddle4534
iddle4535
iddle4729
iddle4959
iddle5011
iddle5040
iddle5041
iddle5060
iddle5097
iddle5098
iddle5099
ch02sb01
ch02sb02
ch02fn01
iddle1257
iddle1258
iddle1676
iddle1679
iddle1749
iddle1750
iddle1909
iddle1910
iddle2004
iddle2005
iddle2022
iddle2023
iddle2032
iddle2401
iddle2518
iddle2519
iddle2522
iddle2534
iddle2547
iddle2580
iddle2581
iddle2715
iddle2864
iddle2865
iddle2866
iddle3522
iddle3523
iddle3638
iddle3889
iddle4338
iddle4385
iddle4386
iddle4395
iddle4396
iddle4481
iddle4482
iddle4732
ch02sb03
ch02fn02
ch02sb04
ch02lev2sec1
ch02list01
iddle1166
iddle1167
iddle2185
iddle2210
iddle2556
iddle2557
iddle2571
iddle2572
iddle2591
iddle2592
iddle2608
iddle2719
iddle2720
iddle2779
iddle2780
ch02sb05
iddle4204
iddle4205
iddle4208
iddle4209
iddle4417
iddle4418
iddle4422
iddle4423
iddle4551
iddle4728
iddle4737
iddle1764
iddle17641
ch02list02
iddle2778
iddle3779
iddle3794
iddle4366
ch02fn03
ch02lev2sec2
ch02fn04
iddle1331
iddle2126
iddle2736
iddle2737
iddle2946
iddle2947
iddle3712
ch02lev2sec3
ch02list03
iddle1035
iddle1036
iddle1037
iddle1038
iddle1144
iddle1145
iddle1146
iddle1149
iddle1240
iddle1241
iddle1330
iddle1411
iddle1412
iddle1413
iddle1414
iddle1417
iddle1418
iddle1419
iddle1420
iddle1421
iddle1422
iddle1423
iddle1559
iddle1560
iddle1561
iddle1680
iddle1681
iddle1682
iddle1683
iddle1704
iddle1770
iddle1771
iddle1772
iddle1773
iddle1774
iddle1864
iddle1865
iddle1866
iddle1867
iddle1868
iddle2464
iddle2465
iddle2503
iddle2504
iddle2716
iddle2840
iddle2841
iddle2842
iddle2843
iddle3635
iddle3639
iddle3640
iddle3736
iddle3793
iddle3876
iddle3877
iddle3878
iddle3879
iddle4102
ch02lev2sec4
iddle4988
iddle4989
iddle4990
ch02sb06
ch02list04
iddle2027
iddle2105
iddle2589
ch02fn05
iddle3156
iddle3157
iddle4207
ch02sb07
iddle4399
iddle4420
iddle4739
iddle1155
iddle1280
iddle1871
iddle1872
iddle2209
ch02fn06
iddle2725
iddle2726
ch02list05
iddle2858
iddle2859
iddle3275
iddle3277
iddle1160
iddle2368
ch02sb08
iddle2587
iddle2588
ch02lev2sec5
iddle2721
iddle2722
iddle2723
iddle2724
iddle2826
iddle2827
iddle2828
iddle2829
iddle2830
iddle2831
iddle3106
iddle3107
iddle3119
iddle3245
iddle3283
iddle3285
iddle4539
iddle4540
iddle4955
ch02list06
iddle2191
iddle3057
iddle3134
iddle3135
ch02lev2sec6
iddle3715
iddle3716
iddle3805
iddle3807
iddle3809
iddle4128
iddle1030
ch02fn07
iddle1815
iddle2014
iddle2015
iddle2218
iddle2478
iddle2479
ch02list07
iddle3092
iddle3093
iddle3150
ch02fn08
iddle4172
iddle4173
iddle4174
iddle4402
iddle1095
iddle1178
iddle1326
iddle1327
ch02sb09
iddle2035
iddle2036
iddle2364
iddle2475
iddle2476
iddle2595
iddle2836
iddle3112
iddle3130
iddle3131
ch02fn09
iddle3367
iddle3368
iddle3595
ch02sb10
iddle3710
iddle3711
iddle4569
ch02fn10
iddle4940
iddle4941
iddle1427
iddle2544
iddle2545
iddle2857
iddle3025
iddle3026
ch02sb11
iddle3276
iddle3497
iddle3498
iddle4206
iddle4419
iddle4738
iddle1474
iddle3531
ch02fig01
iddle4111
iddle4480
ch02list08
iddle2093
iddle2551
iddle2566
iddle2583
iddle2584
ch02sb12
ch02sb13
iddle1031
iddle1753
iddle3209
iddle3211
iddle3362
iddle4237
iddle4408
iddle4409
iddle4548
iddle4560
iddle1141
iddle1428
iddle1429
iddle1430
iddle5105
iddle5107
ch03list01
iddle1939
iddle2146
iddle2618
iddle2619
iddle3398
iddle3399
ch03fn01
iddle3496
iddle3529
iddle3530
iddle3847
iddle3848
iddle3849
iddle3850
iddle3851
iddle3852
ch03sb01
iddle4286
iddle4287
iddle4456
iddle1686
iddle1761
iddle2227
iddle2228
iddle2305
iddle2306
ch03lev2sec1
iddle2558
iddle2559
iddle3409
iddle3410
ch03fn02
iddle4364
iddle4561
iddle4730
ch03list02
ch03list03
ch03lev2sec2
iddle1001
iddle1002
iddle1147
iddle1148
iddle1751
iddle1752
iddle2140
iddle2192
ch03fn03
ch03lev2sec3
iddle2839
iddle3161
iddle3300
iddle3374
iddle3375
ch03fig01
ch03sb02
ch03lev2sec4
iddle5104
iddle1832
ch03fn04
iddle2398
ch03fn05
iddle2610
iddle2640
iddle2641
iddle2642
iddle2919
iddle3383
iddle3384
iddle5064
iddle5122
ch03sb03
iddle3857
iddle4245
iddle4445
iddle4446
iddle4447
iddle4580
ch03fn06
iddle1142
iddle5050
iddle5051
iddle5063
iddle5121
iddle1404
iddle2021
iddle2046
iddle2050
ch03list04
iddle2821
iddle2863
iddle3162
ch03sb04
iddle3717
iddle3718
iddle3719
iddle3720
iddle3721
iddle4107
iddle4108
iddle4133
iddle5046
iddle5066
iddle5124
iddle1068
iddle1070
ch03list05
ch03list07
ch03list06
iddle2212
iddle3791
iddle3792
ch03fn07
iddle1566
iddle1567
iddle1568
iddle2199
iddle2601
iddle2602
ch03lev2sec5
iddle2745
iddle2746
ch03fn08
ch03sb05
iddle4718
iddle4719
iddle1436
iddle1537
iddle1538
iddle1982
ch03list08
iddle2170
iddle2292
iddle2293
iddle1046
iddle1539
iddle1547
iddle1548
iddle4297
iddle4524
iddle4525
iddle1782
iddle1783
iddle1798
iddle1799
iddle1931
iddle4748
iddle4749
iddle4750
ch03fn09
iddle2395
iddle2905
iddle2906
iddle2907
ch03lev2sec6
iddle3035
iddle3601
ch03fn10
iddle4298
iddle4751
iddle4851
ch03lev2sec7
iddle5128
iddle1535
iddle1536
iddle2013
ch03list09
iddle2402
iddle2850
iddle3036
iddle3360
iddle3361
iddle3653
iddle3654
iddle3821
iddle3822
ch03lev2sec8
iddle4356
iddle4357
iddle4358
iddle1546
iddle1549
iddle4756
iddle4757
iddle1965
iddle5052
iddle5053
iddle2448
iddle2449
ch03list10
ch03fn11
iddle1109
iddle4305
iddle4306
iddle4670
iddle4671
iddle4758
iddle4849
iddle5062
iddle1983
iddle2408
iddle2409
iddle2532
iddle2705
iddle2706
iddle2711
iddle2860
iddle2861
iddle3350
ch03sb06
iddle4126
iddle4127
iddle4852
ch03sb07
ch03fn12
iddle2204
iddle2353
iddle2376
iddle2533
iddle2713
ch03list11
iddle1073
iddle1158
ch03fn13
ch03lev2sec9
iddle1726
iddle1920
iddle1921
iddle4496
iddle4497
iddle2343
iddle2344
iddle2350
iddle2351
iddle5016
iddle2527
iddle2568
iddle2712
iddle2742
ch03sb08
ch03lev2sec10
iddle3352
iddle3485
iddle3725
iddle3726
iddle3780
iddle3781
ch03fn14
iddle4124
ch03list12
iddle5119
iddle2149
iddle2851
iddle3061
iddle3062
ch03list13
iddle3732
ch03list14
ch03lev2sec11
iddle1569
iddle1570
ch03fn15
iddle5108
iddle2216
ch03list15
ch03fn16
iddle2150
iddle3723
ch03lev2sec12
iddle2709
iddle2718
iddle2740
iddle2741
iddle2896
ch03sb09
iddle3351
ch03lev2sec13
iddle3722
iddle4033
iddle4034
ch03sb10
iddle4365
ch03lev2sec14
iddle1785
iddle1786
iddle2240
iddle2426
iddle2530
iddle2531
iddle2708
ch03sb11
ch03fn17
iddle2888
iddle3348
ch03lev2sec15
iddle1175
iddle1238
iddle1383
iddle1384
iddle1486
iddle1489
ch03sb12
ch03lev2sec16
iddle1666
iddle1669
iddle1670
iddle1967
iddle4437
iddle4438
iddle4439
ch03sb13
iddle2354
iddle2477
iddle2569
iddle2570
iddle2582
iddle2593
iddle2594
iddle2627
iddle2700
iddle2958
iddle2959
iddle2960
iddle3002
iddle3003
iddle3279
iddle3280
iddle3349
iddle3353
iddle3354
iddle1349
iddle1350
iddle1410
iddle3591
iddle3728
iddle3730
iddle4038
iddle4039
iddle4041
iddle1911
iddle1912
ch04sb01
iddle2031
iddle4216
iddle4235
iddle4236
iddle4555
iddle4583
iddle4584
iddle2609
iddle5071
iddle5072
iddle5120
iddle2862
iddle1154
ch04list01
iddle3345
iddle3363
iddle3364
iddle1542
iddle1562
ch04lev2sec1
iddle3592
iddle3636
iddle3637
iddle2104
iddle4412
iddle4413
iddle4503
iddle4504
iddle4505
iddle2714
iddle4563
iddle3142
iddle3143
iddle1150
iddle1237
iddle1437
iddle3890
iddle3891
iddle1727
ch04sb02
iddle1878
iddle1896
iddle1897
iddle4335
iddle4336
ch04lev2sec2
iddle4421
iddle4429
iddle4571
iddle4572
iddle4755
iddle2546
iddle2567
iddle4960
iddle2855
iddle3273
ch04lev2sec3
iddle3376
iddle3441
iddle3442
iddle3647
iddle4144
iddle1534
iddle4390
iddle4415
iddle4416
iddle2012
iddle2025
iddle5054
iddle2669
iddle2670
iddle2749
iddle3173
ch04fn1
iddle3438
iddle3439
iddle3440
iddle3467
iddle3468
iddle4044
iddle4179
iddle4352
iddle4403
iddle1342
iddle1343
iddle4490
iddle1438
iddle1533
ch04sb03
iddle2048
iddle2049
iddle2153
iddle2520
ch04list02
iddle2747
iddle2748
iddle2956
iddle2957
iddle3059
iddle1842
iddle1843
iddle1946
iddle1947
iddle1948
iddle1949
iddle2051
iddle2052
iddle4217
iddle2374
iddle2512
iddle2638
iddle2639
iddle4745
ch04sb04
iddle2897
iddle2898
ch04lev2sec4
iddle3056
iddle3244
ch04fn02
iddle3708
iddle3709
ch04list03
iddle1632
iddle1638
iddle2159
ch04lev2sec5
iddle5166
iddle5167
iddle5168
iddle2491
iddle2492
iddle2900
iddle3023
iddle3051
iddle3113
iddle3128
iddle3129
iddle3227
iddle3289
iddle1550
iddle1551
iddle1552
iddle3668
iddle3669
iddle1662
iddle1663
iddle1863
ch04fn03
iddle1940
iddle4283
iddle4476
iddle4477
ch04fn04
iddle5080
iddle5082
iddle5089
ch04fn05
ch04list04
iddle4734
iddle2139
iddle2141
iddle2154
ch04list05
ch04lev2sec6
ch04list06
ch04list07
iddle4461
iddle4462
iddle2107
iddle5078
iddle5079
ch04list08
ch04lev2sec7
iddle1553
iddle1554
ch04list09
iddle1941
iddle1151
iddle1335
iddle2215
iddle1862
ch04lev2sec8
iddle2728
iddle2729
iddle2147
iddle2852
ch04list10
iddle2856
iddle3886
iddle3274
ch04sb05
iddle4432
iddle4433
iddle2590
iddle5057
iddle5058
iddle5092
iddle2844
ch04lev2sec9
iddle3733
iddle3734
ch04sb06
ch04lev2sec10
iddle4435
ch04list11
iddle5061
iddle5068
iddle5127
iddle1206
iddle1207
ch04fn06
iddle1555
ch04list12
iddle1571
iddle1942
iddle2171
iddle1164
iddle1344
iddle1345
iddle2698
iddle1722
iddle1723
iddle2010
iddle2011
iddle2028
iddle2029
iddle2288
iddle2391
iddle2392
iddle2414
iddle2415
iddle2416
iddle3666
iddle3667
iddle3593
iddle3594
iddle4961
iddle3737
iddle3738
iddle5081
iddle5093
iddle3997
iddle3998
iddle3999
ch04list13
iddle4492
ch04lev2sec11
iddle4565
iddle4567
iddle4568
iddle1346
iddle1356
iddle1359
ch04list14
iddle2083
iddle1347
iddle1348
iddle1360
iddle1363
iddle1407
iddle1408
iddle1409
iddle2633
iddle2634
ch04list15
iddle1837
iddle1838
iddle1839
iddle1855
iddle1856
iddle1857
iddle1984
iddle3065
iddle2130
iddle2285
iddle2286
iddle2287
iddle2290
iddle2393
iddle2399
iddle2400
ch04lev2sec12
iddle2645
ch04list16
iddle4187
iddle3066
iddle3070
ch04fn07
iddle4292
ch04sb07
iddle1097
iddle4581
iddle4582
iddle1966
iddle2008
iddle2122
iddle2131
iddle5171
iddle2899
iddle2482
ch04fn08
iddle3490
iddle3892
ch04lev2sec13
iddle4564
ch04fn09
iddle1333
iddle1334
iddle1361
iddle1377
iddle1381
iddle1385
iddle1386
iddle1425
iddle1426
iddle1458
iddle1650
iddle1651
ch05lev2sec1
iddle2295
ch05list01
iddle2626
iddle1156
iddle2873
iddle2874
iddle3067
iddle1721
iddle3295
iddle3296
iddle3297
ch05fig01
ch05fn01
iddle2875
iddle4542
iddle4544
iddle4545
ch05list02
ch05list03
iddle5073
iddle5075
iddle5169
iddle5170
ch05list04
ch05lev2sec2
iddle1471
iddle1472
iddle1492
iddle5074
iddle5076
ch05fn02
iddle2774
iddle2233
iddle2300
iddle2885
iddle2886
ch05list05
iddle1378
iddle1379
iddle2876
iddle2877
iddle3076
iddle3239
iddle3240
iddle3241
iddle1659
iddle1660
iddle1661
iddle4060
iddle1717
iddle1718
iddle1719
iddle1720
iddle1809
iddle1935
iddle1936
iddle2019
iddle2020
iddle2024
ch05lev2sec3
iddle2291
iddle2521
iddle5077
iddle2848
iddle2849
iddle2878
iddle2881
ch05sb01
iddle3118
ch05list06
iddle3499
iddle3500
iddle1236
iddle1376
iddle1382
iddle1416
iddle4072
iddle1454
iddle1460
iddle1480
iddle1483
iddle1487
iddle1664
iddle4373
iddle4397
ch05sb02
iddle4549
iddle4550
iddle2121
iddle2574
iddle1461
iddle1490
iddle1491
iddle1493
iddle1494
iddle2624
iddle1558
ch05lev2sec4
iddle1937
iddle2993
iddle2994
iddle3006
iddle3007
iddle3178
iddle3179
iddle3739
iddle3748
iddle4061
iddle2879
iddle2880
iddle2887
iddle2990
iddle3005
iddle3044
iddle3045
iddle3046
iddle3047
iddle3048
iddle3075
iddle3116
iddle3117
iddle3144
iddle3153
iddle1020
iddle1021
iddle4543
iddle1163
iddle3552
iddle3665
iddle1665
iddle1671
iddle1724
iddle1725
iddle4062
iddle4134
iddle4218
iddle4219
iddle4220
iddle4221
iddle4331
iddle4332
iddle4333
iddle4334
ch05fn03
ch05lev2sec5
iddle4474
iddle4475
iddle4502
iddle4969
iddle4970
iddle4971
iddle4972
ch05lev2sec6
iddle5152
iddle2995
iddle3008
iddle3097
iddle1221
iddle1226
iddle3181
ch05list07
iddle1439
iddle1488
iddle1579
iddle1580
iddle3845
iddle3846
iddle1844
iddle1845
iddle1846
iddle3858
iddle3859
iddle1869
iddle1870
iddle1938
iddle2006
iddle2007
iddle4214
iddle4215
iddle2070
iddle2071
iddle4276
iddle4277
iddle4928
iddle1248
iddle1249
iddle1578
iddle3242
iddle3243
iddle3326
iddle3327
iddle3681
iddle3750
iddle3754
iddle2259
iddle2379
iddle2380
ch05sb03
iddle4993
iddle4994
iddle3622
iddle3773
iddle3774
iddle3919
iddle4007
iddle4796
iddle4862
iddle4863
ch05lev2sec7
iddle5156
iddle5157
iddle5158
iddle1121
iddle1390
iddle1391
iddle1543
iddle1544
iddle1705
iddle1706
iddle1840
ch05lev2sec8
iddle2115
iddle2123
iddle2345
iddle2346
ch05list08
iddle2987
ch05list09
iddle3359
iddle3519
iddle3611
iddle3663
ch05lev2sec9
iddle3731
iddle3761
iddle3764
iddle3765
iddle3766
iddle4183
iddle4184
iddle1066
iddle1067
iddle1124
iddle1208
iddle1213
iddle1482
iddle1518
iddle4597
iddle1576
iddle1577
iddle1599
iddle4811
iddle4812
iddle4814
iddle4815
iddle1886
iddle1890
iddle2799
iddle1641
iddle1642
iddle2985
iddle2986
iddle1929
iddle1930
iddle3430
iddle3431
iddle3432
iddle3433
iddle3434
iddle3435
iddle3436
iddle3437
iddle3689
iddle4100
iddle4101
iddle2787
iddle2790
iddle2791
iddle2792
iddle2793
iddle2794
iddle2795
iddle2816
iddle2817
ch05list10
iddle4747
iddle1440
iddle5146
iddle5147
iddle5148
iddle5149
iddle5150
iddle5161
iddle5162
iddle5163
iddle5164
ch05lev2sec10
iddle1582
iddle1624
iddle1625
iddle1629
iddle1653
iddle1654
iddle1728
iddle4125
iddle1903
iddle1904
iddle2026
iddle4327
iddle4328
iddle4329
iddle4330
iddle1136
iddle2194
iddle1291
iddle1300
iddle4463
iddle4464
iddle4469
iddle4515
iddle4516
iddle1397
iddle2446
iddle1441
iddle4974
iddle1607
iddle1697
iddle1700
iddle2932
iddle2933
iddle2934
iddle2935
iddle2936
iddle2197
iddle2225
iddle2423
iddle2430
iddle2432
ch05lev2sec11
iddle3683
iddle3684
iddle2940
ch05list11
iddle4398
iddle4585
iddle4586
iddle4587
iddle4588
iddle4589
iddle4590
ch05list12
iddle3982
iddle4019
iddle2039
iddle2040
iddle2156
iddle2226
iddle2261
iddle2436
iddle4424
iddle4425
iddle4604
iddle4605
iddle4654
iddle4655
iddle5144
iddle5145
iddle1319
iddle1320
iddle1442
ch05list13
iddle1701
ch05lev2sec12
iddle2252
iddle2253
iddle1039
iddle1060
iddle1061
ch05fn04
iddle1189
iddle1190
iddle1191
iddle1192
iddle1194
iddle1197
iddle1204
iddle1205
iddle1209
iddle1210
iddle1242
iddle1243
iddle1259
iddle1260
iddle1374
iddle1375
iddle2701
iddle2702
iddle1443
iddle1741
ch05lev2sec13
iddle2067
iddle3551
iddle3612
iddle3613
iddle2443
iddle2444
iddle2445
iddle3841
iddle3842
iddle3936
iddle3937
iddle4025
iddle4136
iddle4145
iddle4146
iddle4150
iddle2882
iddle2883
iddle2937
iddle1128
iddle1129
ch05list14
iddle1288
iddle1289
iddle1328
iddle1329
iddle3284
iddle4880
iddle4881
iddle3357
iddle1444
iddle3452
iddle3453
iddle3555
iddle3556
iddle3609
iddle4123
iddle4148
iddle4149
iddle1994
iddle2091
iddle2239
ch05fn05
iddle4574
iddle4575
iddle4576
iddle4577
iddle4578
iddle4890
iddle2961
iddle2962
ch05list15
iddle3465
iddle3466
iddle1459
iddle3979
iddle3980
iddle4080
iddle4081
iddle1734
iddle1735
iddle1742
iddle4138
iddle4213
iddle4288
iddle4289
iddle2097
iddle2098
iddle2113
iddle2135
iddle4478
iddle4479
ch05list16
iddle4661
iddle4662
iddle1051
iddle1285
iddle3192
ch05fig02
iddle3489
iddle2137
ch05fig03
ch05list17
iddle4541
ch05fig04
iddle1159
iddle1283
iddle1284
ch05list18
iddle2435
iddle1424
iddle3866
iddle4032
iddle2138
ch05list19
ch05list20
iddle1024
iddle1025
ch05sb04
iddle2136
iddle1415
iddle2437
iddle1464
ch05fn06
iddle1657
iddle1658
iddle2114
iddle1898
iddle1042
iddle1045
iddle1961
iddle1112
iddle1113
iddle2009
iddle1198
iddle1199
iddle1255
iddle1357
iddle1358
iddle2352
iddle1455
iddle2505
iddle2707
iddle1841
iddle1847
iddle2847
iddle1880
iddle1884
iddle1885
iddle1925
iddle1926
ch06lev2sec1
iddle3074
ch06list01
iddle2249
iddle2307
iddle2308
iddle2381
iddle2382
iddle2461
iddle2731
iddle2732
ch06fn01
iddle2180
iddle4410
iddle4411
ch06lev2sec2
iddle2251
ch06list02
iddle3458
iddle3583
iddle2488
iddle2489
iddle2679
iddle2680
iddle3864
iddle3865
iddle3967
iddle3968
iddle3983
iddle3984
iddle4191
iddle4274
iddle4275
iddle4317
iddle4318
iddle1730
iddle4394
ch06lev2sec3
iddle1125
iddle1126
iddle4601
iddle4602
iddle4603
iddle4606
iddle4618
iddle4619
iddle4622
iddle4829
iddle4830
iddle3589
iddle3590
iddle2203
iddle4874
iddle4875
iddle3596
iddle3597
iddle5165
iddle3926
iddle3969
iddle1563
iddle1564
iddle4096
iddle4097
ch06fn02
iddle4163
iddle4164
iddle4169
iddle1732
iddle1765
iddle1766
iddle1767
iddle1768
iddle1769
iddle1836
iddle4301
iddle4302
iddle4623
iddle4820
iddle4821
iddle4876
iddle4877
iddle3459
iddle3600
iddle1134
iddle1135
ch06list03
iddle4098
iddle2981
iddle2982
iddle3199
ch06lev2sec4
iddle3420
iddle3421
iddle1849
iddle1850
iddle3491
iddle3492
iddle1922
iddle1923
iddle1924
iddle4624
iddle4650
iddle4651
iddle4764
iddle3749
iddle4831
ch06list04
iddle3910
iddle3911
iddle3931
iddle2254
iddle2255
ch06list05
iddle4353
iddle4354
iddle4355
iddle2761
iddle2762
ch06lev2sec5
ch06list06
iddle2976
iddle2977
iddle4765
iddle4786
iddle47861
iddle4787
iddle4832
iddle4838
iddle4997
iddle4998
iddle5154
ch06fn03
iddle3598
iddle3623
iddle3685
ch06sb01
iddle1597
iddle1598
ch06lev2sec6
iddle2193
iddle2202
iddle2250
iddle3921
iddle3928
iddle2219
iddle2375
iddle2460
iddle2524
iddle2525
ch06fn04
iddle4797
iddle1733
iddle3582
iddle2267
iddle2268
iddle2269
iddle2297
ch06fn05
ch06lev2sec7
iddle3486
iddle3619
iddle3625
iddle3746
iddle3747
iddle3775
iddle3895
iddle3920
iddle4620
ch06list07
iddle3572
iddle3573
iddle3628
iddle2274
iddle2275
iddle2456
iddle2457
iddle2458
iddle4792
iddle4794
iddle4009
iddle2758
iddle2759
ch06list08
iddle5159
iddle2973
iddle2974
ch06lev2sec8
iddle1089
ch06fn06
iddle4799
iddle1256
iddle3599
iddle2128
iddle3825
iddle3826
iddle3929
iddle4017
iddle4018
iddle1852
iddle1853
iddle1854
iddle4252
iddle4267
iddle4269
iddle1913
iddle1914
iddle4400
iddle4401
iddle2234
iddle2248
iddle4635
iddle4641
iddle4672
iddle4673
iddle2563
ch06list09
iddle1445
ch06lev2sec9
iddle3830
iddle2954
iddle1710
iddle4265
iddle4266
iddle3450
iddle3581
ch06list10
iddle2151
ch06lev2sec10
iddle4090
iddle4091
iddle1292
iddle1315
iddle1316
ch06fn07
iddle2678
iddle1432
iddle1433
iddle4296
iddle4607
iddle4659
iddle4660
iddle4784
iddle1927
iddle1928
iddle4886
iddle4887
iddle4916
iddle4917
iddle4920
iddle4921
iddle4932
iddle4933
iddle1012
iddle1013
ch06list11
iddle3446
iddle3447
iddle2179
iddle2229
iddle2230
iddle1290
iddle2425
iddle2427
iddle3966
ch06list12
iddle4167
ch06lev2sec11
iddle2978
iddle2979
iddle2980
iddle2419
iddle2428
iddle2429
iddle2440
iddle5036
ch06lev2sec12
iddle3867
iddle3868
iddle3869
iddle3870
iddle3981
iddle3985
iddle3991
iddle3992
iddle3995
iddle3996
iddle4016
iddle3298
iddle1225
ch06list13
iddle4633
iddle4634
iddle4636
iddle4645
iddle4646
iddle2635
iddle2636
iddle3729
iddle1400
iddle1401
iddle1405
iddle1406
iddle4897
iddle4898
iddle3871
iddle5114
iddle5115
iddle4040
ch06sb02
ch06lev2sec13
iddle3445
iddle3451
iddle3484
iddle2165
iddle2238
iddle2265
iddle2266
iddle4647
ch06list14
iddle4858
ch06lev2sec14
iddle4067
iddle2643
iddle2644
iddle4312
ch06list15
iddle2116
iddle4628
iddle4629
ch06lev2sec15
iddle3678
iddle3679
iddle3680
iddle1317
iddle3753
iddle2169
iddle1934
ch06fn08
iddle4630
iddle4631
iddle4632
iddle4639
iddle4640
ch06fn09
ch06lev2sec16
iddle3517
iddle3518
ch06list16
iddle3958
iddle3959
iddle4241
iddle4242
iddle3633
iddle3634
ch06list17
iddle3933
iddle1043
iddle1044
iddle1296
iddle1297
iddle1298
iddle1299
ch07fn01
iddle1314
iddle1351
iddle1354
iddle4888
iddle4929
iddle4930
iddle4962
iddle4963
iddle1565
iddle1643
iddle1114
iddle1115
iddle2034
iddle1310
iddle1313
iddle2309
iddle2735
iddle2796
iddle2811
iddle2812
iddle2963
iddle2964
iddle2965
iddle2966
iddle2967
iddle2968
iddle2969
iddle2970
iddle2971
iddle2975
iddle2041
iddle2042
iddle2060
iddle2221
iddle2222
iddle2223
iddle2276
iddle2277
iddle2278
iddle2301
iddle2302
iddle2303
iddle2304
iddle2369
iddle2370
ch07list01
iddle1214
iddle1223
iddle1304
iddle3650
iddle3651
ch07list02
iddle3798
ch07lev2sec1
iddle4889
iddle4246
iddle4251
ch07sb01
iddle4467
iddle4514
iddle2157
iddle3574
iddle4609
iddle4674
iddle4675
iddle4676
iddle4684
iddle2371
iddle2394
iddle4775
iddle4776
iddle4828
iddle4841
iddle4842
iddle4843
iddle4844
iddle2538
iddle2539
iddle5014
iddle3988
iddle3989
ch07list03
iddle4196
iddle4197
iddle2785
iddle2797
iddle2820
iddle4250
ch07list04
ch07sb02
iddle4610
iddle4612
iddle4613
iddle1224
iddle1309
iddle1312
iddle4891
iddle4892
iddle4977
iddle4979
iddle5003
iddle5024
iddle5025
iddle5116
iddle5117
iddle3751
iddle2092
ch07sb03
iddle1848
ch07list05
iddle4448
ch07lev2sec2
iddle4725
iddle4779
iddle4781
iddle1311
iddle2781
iddle2782
iddle2813
iddle2814
iddle2818
iddle1352
iddle1353
iddle3557
iddle3576
iddle2158
iddle3752
iddle1684
iddle1685
ch07sb04
iddle3873
iddle3874
iddle3875
iddle1775
iddle1776
ch07lev2sec3
iddle4726
iddle2809
iddle2810
ch07list06
iddle4470
iddle1215
iddle1216
iddle4517
iddle2378
iddle2540
iddle4727
ch07sb05
iddle2783
iddle2784
iddle2801
iddle2802
iddle2803
iddle3575
iddle2815
iddle3584
iddle3585
iddle1992
iddle1993
ch07list07
ch07lev2sec4
iddle3701
iddle3702
iddle2542
iddle3953
iddle3954
iddle4611
iddle3975
iddle2786
iddle4008
iddle4657
iddle4658
iddle2805
iddle2819
iddle4833
iddle3174
iddle3175
ch07list08
iddle4471
iddle1301
iddle3604
ch07fn02
iddle3951
iddle3952
iddle4014
iddle4015
ch07lev2sec5
ch07list09
iddle2173
iddle4518
iddle2422
iddle2541
iddle2909
iddle2910
iddle2911
iddle1217
iddle1308
ch07list10
ch07sb06
ch07lev2sec6
iddle2272
iddle1132
iddle1306
iddle1307
iddle4894
iddle4457
iddle4458
ch07lev2sec7
iddle2596
iddle2597
iddle4720
iddle4721
iddle4722
iddle2788
iddle2789
iddle2806
iddle2807
iddle2016
ch07list11
iddle2282
iddle2294
iddle2424
iddle1364
iddle2672
iddle2673
iddle2681
iddle2682
iddle2808
iddle2832
iddle4472
iddle2902
iddle2903
iddle2904
iddle3990
iddle3042
iddle3053
iddle3100
iddle3108
iddle2030
iddle3299
iddle2166
ch07sb07
ch07lev2sec8
ch07fn03
ch07list12
iddle2605
iddle2606
iddle2772
iddle2773
iddle4931
iddle2822
iddle2823
iddle4118
iddle4119
iddle2983
iddle4325
iddle4326
iddle3069
iddle3169
iddle3170
ch07list13
iddle4595
iddle4596
iddle3443
iddle2094
iddle2134
iddle2183
iddle4859
iddle1011
iddle2438
iddle3872
iddle1153
iddle1336
iddle4194
iddle4195
iddle4198
iddle4199
iddle4258
iddle4262
ch07list14
iddle2921
iddle4487
iddle4493
iddle4494
iddle4495
ch07lev2sec9
iddle4788
iddle4791
ch07list15
iddle4816
iddle4817
iddle2132
iddle2270
iddle2271
iddle2462
iddle2463
ch07list16
ch07lev2sec10
ch07list17
iddle2018
iddle2133
iddle3782
ch07lev2sec11
iddle3964
iddle3965
iddle4042
iddle4043
ch07fn04
iddle1008
iddle1009
iddle4253
iddle4268
iddle4271
ch07list18
ch07list19
iddle1337
iddle1338
iddle1339
iddle2125
ch07list20
iddle2972
ch07lev2sec12
ch07fn05
iddle3560
iddle3561
iddle3562
iddle3566
ch07fn06
iddle2106
iddle2124
iddle2273
iddle2355
ch07list21
iddle4260
iddle4261
iddle4263
iddle2760
iddle3565
iddle4485
iddle4486
iddle4489
iddle3770
iddle3771
iddle1004
iddle1005
ch07list22
iddle4995
iddle2207
iddle4264
iddle3563
iddle3564
iddle4488
iddle1821
iddle1822
iddle2152
iddle4249
iddle4270
iddle2217
iddle2236
iddle2237
iddle4426
iddle4449
iddle4450
iddle4656
iddle2685
iddle2686
ch07fn07
iddle4780
iddle2953
iddle5118
ch07list23
ch07lev2sec13
ch07fn08
iddle3784
iddle3785
ch07list24
iddle4023
iddle4024
iddle4026
iddle4027
iddle4050
iddle4051
ch07list25
iddle2235
iddle2315
iddle2316
ch07sb08
iddle5153
iddle1010
iddle4681
iddle4682
iddle4744
iddle4782
iddle2929
ch07lev2sec14
iddle2955
iddle4978
iddle4981
iddle5004
iddle5005
iddle5013
iddle1355
iddle2208
iddle2231
iddle2232
iddle2241
iddle2242
iddle2523
ch07list26
ch07lev2sec15
iddle2356
iddle2357
iddle3167
iddle3168
iddle4529
iddle2648
iddle2649
iddle2650
iddle2651
iddle3632
ch07lev2sec16
ch07sb09
iddle4773
iddle4774
iddle4785
iddle1745
iddle4853
iddle4854
iddle2922
iddle2924
iddle4999
iddle5002
ch07fn09
ch07sb10
iddle3419
iddle2358
iddle4508
iddle4509
iddle2603
iddle2604
iddle1985
iddle4166
iddle1540
iddle4980
iddle2923
iddle5000
iddle4247
iddle4248
iddle4254
iddle4255
iddle4256
iddle4259
iddle4293
iddle1707
iddle1708
ch08fn01
iddle1881
iddle1882
iddle1905
iddle1906
iddle3293
iddle3294
iddle2528
iddle4731
iddle2243
iddle2244
iddle2245
iddle2247
iddle2260
iddle4975
ch08sb01
iddle3922
iddle1818
ch08lev2sec1
iddle1883
iddle1963
iddle1964
iddle2717
iddle4257
ch08sb02
iddle1475
iddle2493
iddle2494
iddle2526
ch08list01
iddle4766
iddle1817
iddle3577
iddle3578
iddle3580
ch08lev2sec2
iddle3621
iddle1220
iddle1234
iddle1235
iddle4468
iddle2200
iddle2263
iddle2264
iddle1431
iddle2517
iddle1932
iddle1933
iddle4615
iddle4616
iddle4621
iddle3942
iddle3943
iddle4752
iddle4795
iddle4375
iddle1525
iddle1526
iddle2676
iddle2677
iddle1672
iddle1673
iddle2763
iddle4608
iddle4617
iddle1729
ch08lev2sec3
iddle4648
iddle4649
iddle1784
iddle4824
iddle1950
iddle4850
ch08fn02
iddle4922
iddle4923
ch08fn03
iddle1251
iddle2246
iddle2296
iddle1476
ch08list02
iddle4374
iddle3540
iddle3541
iddle1574
iddle3631
iddle1674
iddle3897
iddle3924
iddle3955
iddle3956
ch08lev2sec4
iddle4823
iddle4319
iddle1222
iddle3208
iddle1239
iddle1252
ch08fn04
iddle3542
iddle3579
iddle4637
iddle4638
iddle3614
iddle3615
iddle3616
iddle3630
iddle4803
iddle4910
iddle4911
iddle2926
iddle2927
iddle2930
iddle2931
iddle3930
iddle3938
iddle4991
iddle4992
iddle3971
iddle3972
iddle5129
iddle5130
iddle5131
iddle3182
iddle4307
iddle4308
iddle1006
iddle1007
iddle4313
iddle4314
ch08sb03
iddle1059
ch08fn05
iddle3617
iddle3618
iddle2383
iddle2384
iddle3627
iddle1250
iddle1295
iddle3768
iddle3769
ch08fn06
iddle4665
iddle4666
ch08lev2sec5
iddle3932
iddle4763
iddle4801
iddle4834
iddle4855
iddle4987
iddle4303
iddle2989
iddle4315
iddle4316
iddle1952
iddle1953
iddle3196
iddle3197
iddle4642
iddle4643
iddle4644
iddle4683
ch08list03
iddle3664
iddle4804
iddle4835
iddle4836
iddle4856
iddle4903
iddle4904
iddle3767
iddle4925
iddle4926
iddle1527
iddle1528
iddle2607
iddle3893
iddle3894
ch08lev2sec6
iddle3973
iddle1731
iddle1736
iddle1737
iddle1738
iddle1831
ch08list04
iddle2090
ch08list05
iddle3306
iddle2298
iddle2299
iddle1032
iddle1033
iddle4599
ch08list06
iddle3532
iddle2459
iddle3586
iddle4864
iddle3755
iddle3756
iddle3828
iddle3829
iddle3832
iddle3833
iddle4996
iddle1477
iddle1529
iddle1530
iddle3925
ch08lev2sec7
iddle1575
iddle5160
iddle4052
iddle4053
iddle1739
iddle1740
ch08list07
iddle1165
iddle2142
iddle2144
iddle4598
iddle4600
iddle4805
ch08list08
iddle4865
iddle3923
iddle4142
iddle2143
ch08lev2sec8
iddle2289
iddle3171
ch08list09
iddle3548
iddle3549
iddle3550
iddle4771
iddle4772
iddle3587
iddle3588
iddle2652
iddle3626
iddle4846
iddle2764
iddle5001
iddle1062
iddle1063
ch08list10
iddle4116
iddle4117
iddle4139
iddle3172
iddle3191
iddle3253
ch08sb04
iddle4304
iddle3620
ch08list11
iddle1915
iddle4802
iddle4845
iddle4866
iddle2206
ch08list12
ch08lev2sec9
ch08fn07
iddle4443
iddle4444
iddle2577
iddle2578
ch08list13
iddle2884
iddle4793
iddle4857
iddle4914
iddle5012
ch08list14
iddle3455
iddle2163
iddle3804
iddle1699
iddle4115
iddle4165
iddle2727
ch08list15
iddle2145
iddle2176
iddle2213
ch08list16
iddle3454
iddle4967
iddle4968
ch08list17
iddle3740
iddle3741
iddle2099
iddle2941
iddle2184
iddle2214
ch08list18
iddle4427
iddle4428
iddle1110
iddle4736
iddle3976
iddle3977
iddle3978
iddle4112
iddle4113
iddle4114
iddle4956
iddle4957
iddle4958
iddle4360
iddle4361
iddle2164
iddle4483
iddle4484
iddle3827
iddle3831
iddle1907
iddle1908
iddle1981
iddle1545
iddle1630
iddle1631
iddle1644
iddle1645
ch09fn01
iddle5039
iddle1807
iddle2486
iddle2495
iddle2496
ch09lev2sec1
iddle4677
iddle1179
iddle4893
ch09lev2sec2
iddle3269
iddle3270
iddle1541
iddle1621
iddle1622
iddle1623
ch09sb01
iddle3783
iddle2998
iddle2999
iddle2059
iddle2072
iddle2073
iddle3127
iddle3225
iddle3226
iddle2187
iddle3287
iddle3288
iddle2256
iddle3408
iddle2335
iddle4243
iddle4244
iddle2483
iddle2484
iddle2485
iddle4299
iddle4300
ch09fig01
ch09list01
iddle2119
iddle2188
iddle2189
iddle2996
iddle4818
iddle4819
ch09list02
iddle2257
iddle3365
iddle3366
ch09list03
iddle3528
iddle3652
iddle3699
iddle1026
iddle1027
iddle1041
iddle4753
iddle3801
iddle3802
ch09fig02
iddle5087
iddle5088
iddle4161
iddle4162
iddle1626
iddle1627
iddle1628
ch09list04
iddle4519
iddle4522
iddle4523
iddle4526
ch09list05
iddle2063
iddle2064
iddle2065
iddle2120
iddle4754
iddle2258
iddle2262
iddle4954
iddle2366
iddle2367
iddle5023
ch09lev2sec3
iddle2490
iddle1305
ch09list06
iddle4520
iddle4521
iddle4627
ch09lev2sec4
iddle2997
iddle3000
iddle3220
iddle3221
iddle4924
iddle2336
iddle2337
iddle1398
iddle1399
ch09lev2sec5
iddle1757
iddle2420
iddle2421
iddle3904
iddle3905
ch09list07
iddle4339
iddle4340
iddle2433
iddle2434
ch09list08
iddle4625
iddle4626
iddle2646
iddle2647
iddle4909
iddle2079
iddle5084
iddle5085
iddle5086
ch09lev2sec6
iddle3231
iddle3319
iddle3320
iddle2439
iddle1446
iddle3700
iddle1469
iddle1470
ch09lev2sec7
iddle1667
iddle1668
iddle1759
iddle1760
iddle1762
iddle5026
iddle5027
iddle4226
ch09sb02
iddle2108
iddle4530
iddle4531
iddle4918
iddle4919
iddle3004
iddle1465
iddle1466
iddle1467
iddle1468
iddle3234
iddle3235
iddle3238
iddle1633
iddle1791
iddle1792
iddle1793
iddle1794
iddle1820
iddle1851
iddle1951
iddle1975
iddle4965
iddle4966
iddle2043
iddle2044
iddle2045
ch10lev2sec1
iddle1556
iddle1557
ch10fig01
iddle2319
iddle2320
iddle2372
iddle2373
ch10sb01
iddle4238
iddle1780
iddle1781
iddle1810
iddle1811
iddle4345
iddle4346
iddle4347
ch10list01
ch10lev2sec2
iddle4733
ch10list02
iddle3017
iddle3018
iddle3019
iddle3020
iddle3021
iddle3022
iddle3085
iddle3086
iddle2515
iddle2552
iddle5017
iddle5018
iddle5083
iddle5094
iddle5095
iddle5096
iddle1806
iddle3648
iddle2915
iddle3800
iddle1978
ch10list03
iddle3944
iddle3945
iddle3946
iddle3947
iddle3948
iddle3124
iddle3125
iddle2127
iddle4037
iddle3405
iddle3406
ch10fn01
ch10fn02
ch10list04
iddle2628
iddle2733
iddle2734
ch10lev2sec3
iddle1069
iddle4882
iddle4883
iddle4884
iddle4885
iddle3126
iddle3054
iddle3407
ch10sb02
iddle1447
iddle2109
ch10lev2sec4
iddle1592
iddle1648
iddle1803
ch10fn03
ch10list05
iddle1152
iddle1646
iddle1647
ch10sb03
iddle2513
iddle2514
iddle1816
iddle4459
iddle2110
iddle4905
iddle2195
ch10lev2sec5
ch10list06
iddle3081
iddle3122
iddle3123
iddle3369
iddle3370
iddle2516
iddle4694
ch10lev2sec6
iddle1802
iddle1804
iddle1805
iddle1814
iddle1819
iddle1086
iddle1087
iddle3713
iddle3714
iddle2111
ch10lev2sec7
iddle3909
iddle2196
iddle2317
iddle2318
iddle4073
ch10fn04
iddle1795
iddle1796
iddle1833
ch10list07
iddle3043
iddle3158
iddle3629
ch10lev2sec8
iddle3155
iddle1649
ch10fn05
iddle4376
ch10sb04
ch10lev2sec9
iddle4460
ch10lev2sec10
iddle4800
iddle4837
iddle4927
iddle2330
iddle2331
iddle2585
iddle2586
iddle4362
iddle4363
iddle2053
iddle2054
iddle2889
iddle1092
ch11fn01
iddle4767
iddle4768
iddle2283
iddle2284
iddle1253
iddle4976
iddle1617
iddle1655
iddle1656
iddle3660
iddle3661
ch11fn02
iddle1916
iddle1917
iddle3011
iddle3012
iddle3013
ch11lev2sec1
iddle4095
iddle4131
iddle4132
ch11sb01
iddle1714
iddle1715
iddle4367
iddle4368
iddle4369
iddle4370
iddle4371
iddle3558
iddle3559
iddle3662
iddle3786
iddle3787
iddle3962
iddle3963
iddle3993
iddle3994
iddle4806
iddle4807
ch11lev2sec2
iddle1695
iddle1696
iddle2575
iddle3252
iddle2660
iddle2661
iddle2662
iddle2663
iddle2664
iddle2665
iddle2666
iddle2668
ch11sb02
iddle1918
iddle1919
iddle3482
iddle3483
iddle1083
iddle1084
iddle1085
iddle1177
iddle2058
iddle1200
iddle4808
iddle4809
iddle3185
iddle3906
iddle1462
iddle1463
iddle3934
iddle3935
iddle3960
iddle3961
iddle3393
iddle4055
iddle3501
iddle3502
iddle3503
iddle3504
iddle3505
iddle3506
iddle1687
iddle1688
iddle1689
iddle1690
iddle1691
iddle3524
iddle2562
iddle1823
iddle1824
iddle1825
iddle1826
iddle1827
iddle1829
iddle1830
iddle2667
ch11sb03
iddle1891
iddle1892
iddle1893
iddle1894
iddle1895
iddle1077
iddle1078
iddle1079
iddle1080
iddle1954
iddle1955
iddle1956
iddle1957
iddle1958
iddle3912
iddle3913
iddle2055
iddle2056
iddle2057
iddle2943
iddle2944
iddle2945
iddle4054
iddle4074
iddle4092
iddle4093
iddle4094
iddle3236
iddle3237
iddle2362
iddle2385
iddle2386
iddle2387
iddle2388
iddle2389
iddle2390
iddle3391
iddle2450
iddle2451
iddle2452
iddle2453
iddle2454
iddle2455
iddle2498
iddle2499
iddle2500
iddle2501
ch11fig01
iddle2553
iddle2554
iddle2564
iddle2565
iddle3525
iddle3546
iddle3547
iddle2630
iddle2671
iddle2750
iddle2751
iddle2752
iddle2753
iddle2754
iddle2755
iddle2867
iddle2868
iddle3884
iddle3885
iddle4048
iddle4049
iddle4077
iddle3166
iddle3183
iddle3184
iddle3193
iddle5035
iddle5037
iddle5038
iddle1028
iddle1029
ch11list01
iddle3246
iddle3247
iddle3248
iddle3249
iddle3250
iddle1082
iddle3344
iddle3392
ch11sb04
iddle1777
iddle1778
iddle3469
iddle3470
ch11lev2sec3
iddle3567
iddle3568
iddle3569
iddle3570
iddle3571
iddle3606
iddle3690
ch11fig02
iddle3703
iddle3704
iddle3705
iddle3742
iddle3880
iddle3881
iddle3898
iddle3899
iddle3900
iddle3901
iddle3902
iddle3903
iddle3916
iddle3917
iddle3918
iddle4005
iddle4006
iddle4010
iddle4011
iddle4012
ch11lev2sec4
iddle4120
iddle4121
iddle3186
iddle3189
iddle3190
iddle4282
iddle3389
iddle3390
ch11fn03
iddle4337
iddle1608
iddle1620
iddle3456
iddle3457
iddle3472
iddle3513
iddle4442
iddle4451
iddle4452
iddle4453
iddle4454
iddle4455
iddle2220
iddle1286
iddle1287
iddle3694
iddle2410
iddle4667
iddle4668
iddle4669
iddle4687
iddle4688
iddle4689
iddle4690
iddle4691
iddle4692
ch11lev2sec5
iddle3939
iddle3940
iddle1611
iddle1612
iddle2579
iddle1692
iddle4937
iddle4938
iddle4945
iddle4180
iddle4181
iddle4185
iddle4186
ch11list02
iddle5109
iddle5110
iddle1088
iddle2471
iddle2472
iddle1195
iddle1212
iddle1281
iddle1282
ch11lev2sec6
iddle4652
iddle4653
iddle1604
iddle3986
iddle3987
iddle2467
iddle2468
iddle4951
iddle1057
iddle4229
iddle4230
iddle5030
iddle5031
iddle5032
iddle5033
iddle3762
iddle3763
ch11fn04
iddle4272
iddle4273
ch11list03
iddle1182
iddle1183
iddle2047
iddle1366
ch11fn05
iddle4078
iddle4079
ch11sb05
iddle4176
iddle3101
iddle3102
iddle3147
iddle3152
iddle2334
ch11fn06
ch11lev2sec7
iddle1227
iddle1233
iddle5034
iddle1995
iddle1996
iddle4082
iddle4177
iddle4178
iddle4182
ch11sb06
iddle2916
iddle2925
iddle1593
iddle1594
iddle3887
iddle3888
iddle4872
iddle4873
ch11fn07
ch11sb07
ch11lev2sec8
iddle4058
iddle3194
iddle3206
iddle3207
iddle2550
iddle2560
iddle2561
ch11fn08
iddle3476
iddle3509
iddle4349
ch11list04
iddle4501
iddle3854
iddle3855
iddle1081
iddle3071
iddle1161
iddle1162
iddle3090
ch11list05
iddle4762
iddle2576
iddle3304
iddle3386
iddle3387
iddle2078
ch11fn09
iddle3009
iddle3010
ch11lev2sec9
iddle3082
iddle3083
iddle4546
iddle1860
iddle1861
iddle4554
iddle4558
iddle4559
iddle4579
iddle2548
iddle2549
iddle1367
iddle3388
iddle2082
iddle3516
iddle4946
iddle5006
iddle5007
iddle5112
iddle1808
iddle3080
ch11list06
iddle3145
ch11list07
iddle3290
iddle3291
iddle3292
iddle4065
iddle4066
ch11lev2sec10
iddle1380
iddle4341
iddle4342
iddle2469
iddle1747
iddle1748
iddle4510
iddle4511
ch11fn10
ch11fn11
iddle2730
iddle4071
iddle4109
iddle4110
ch11lev2sec11
iddle4825
iddle4826
iddle2177
ch11list08
iddle3073
iddle1138
iddle3103
iddle3146
iddle3148
iddle2340
iddle2341
iddle5142
iddle2629
iddle2653
iddle2654
iddle2655
iddle4907
iddle4908
iddle2837
ch11fn12
iddle4491
ch11lev2sec12
iddle3114
iddle3149
iddle3151
iddle3154
iddle4735
iddle2186
iddle1090
iddle1091
iddle2342
ch11lev2sec13
iddle4348
iddle4350
iddle4434
iddle2466
iddle4473
iddle3803
iddle2656
iddle2657
iddle1591
iddle1713
iddle4068
iddle1074
iddle1075
iddle5059
iddle3094
iddle3095
ch11lev2sec14
iddle4351
ch11fn13
iddle3259
iddle3260
ch11fn14
iddle4499
iddle4500
iddle1473
iddle3526
iddle3797
ch11fn15
ch11sb08
iddle2684
iddle1071
iddle1072
iddle4075
iddle2869
iddle3078
iddle3079
iddle1481
ch11fig03
iddle3187
iddle3251
iddle3257
iddle3258
iddle3473
iddle3474
iddle3475
iddle3692
iddle3693
ch11fn16
iddle1615
iddle1616
iddle1618
iddle4064
iddle3355
iddle3356
iddle3358
iddle2502
iddle3527
iddle2625
iddle3607
iddle3608
iddle3610
iddle4770
iddle4076
iddle3180
iddle4943
iddle4944
iddle4949
iddle4950
iddle2674
iddle2675
iddle5111
iddle1479
iddle2615
ch12fn01
iddle1678
iddle4063
iddle1211
iddle1266
iddle1268
iddle1272
iddle5113
iddle4536
iddle1677
ch12fn02
iddle1702
iddle2310
iddle2311
ch12list01
iddle3414
iddle1779
iddle3461
iddle3462
iddle4188
iddle4189
iddle2632
ch12lev2sec1
iddle3724
iddle2085
iddle2086
ch12list02
iddle1218
iddle1219
iddle1269
ch12lev2sec2
iddle4190
ch12fn03
iddle3536
iddle1899
iddle3672
iddle3673
iddle3029
iddle3030
iddle2089
iddle4047
ch12list03
iddle3537
ch12lev2sec3
ch12sb01
iddle2845
iddle3949
iddle4059
iddle4693
iddle4709
iddle4714
iddle4140
iddle4141
iddle4151
iddle1340
iddle1341
iddle2800
ch12fn04
ch12list04
iddle4697
iddle4698
iddle4700
iddle4707
iddle4708
iddle4712
iddle3882
iddle3883
iddle4868
iddle5010
iddle4170
iddle2573
iddle1743
ch12sb02
ch12list05
iddle4663
iddle4664
iddle2161
ch12list06
iddle5008
iddle5009
ch12lev2sec4
iddle2600
iddle3688
iddle4723
iddle4724
iddle3401
iddle3402
iddle1293
iddle1294
ch12fn05
iddle4046
iddle3790
ch12lev2sec5
ch12list07
iddle1986
iddle2103
ch12list08
iddle2160
ch12list09
iddle4713
iddle4717
ch12lev2sec6
iddle1101
iddle1102
iddle2631
iddle2846
ch12list10
iddle2084
iddle2951
iddle2952
iddle2198
iddle1193
iddle3198
iddle4678
iddle4679
ch12lev2sec7
iddle3342
iddle3343
iddle4789
iddle4790
ch12list11
iddle2756
iddle2757
iddle1744
ch12fig01
iddle3896
iddle1246
iddle1247
iddle3914
iddle3915
iddle1264
iddle1265
iddle1270
iddle1271
iddle3927
iddle2162
iddle1050
iddle1076
ch12list12
iddle1123
iddle2770
iddle2771
iddle1712
ch12list13
iddle2417
iddle2418
iddle2890
ch12lev2sec8
iddle3624
iddle1585
iddle1586
ch12fn06
iddle2080
iddle4680
iddle2205
iddle4783
ch12fig02
ch12lev2sec9
iddle4942
iddle1973
iddle1974
iddle1402
iddle1403
ch12fig03
iddle3538
ch12fig04
iddle4716
iddle4798
ch12lev2sec10
iddle3471
ch12lev2sec11
iddle2988
iddle4309
iddle3687
iddle2473
iddle2474
ch12fn07
ch12fig05
iddle1392
iddle1393
iddle1394
iddle2324
iddle2325
iddle4086
iddle4695
iddle4696
iddle4699
iddle4701
ch12lev2sec12
iddle4912
iddle4913
iddle4310
iddle4311
iddle3188
ch12fn08
iddle5021
iddle5022
iddle1976
iddle1977
iddle1434
iddle1435
iddle4056
iddle4057
iddle2441
iddle2442
iddle1609
iddle1610
ch12lev2sec13
iddle3649
iddle3744
iddle3745
iddle4870
iddle4871
iddle3308
iddle3309
iddle4498
iddle3957
iddle2658
iddle2659
ch12lev2sec14
iddle4686
iddle1787
iddle1788
iddle3539
ch12sb03
iddle4153
iddle4154
iddle4155
iddle4156
iddle4614
iddle2914
iddle4901
iddle4902
iddle5069
iddle5070
iddle3256
ch12lev2sec15
iddle3380
iddle3381
iddle3382
iddle1273
iddle1274
iddle1275
iddle1276
iddle1277
ch12lev2sec16
iddle1369
iddle1370
ch12fn09
iddle4702
iddle4703
iddle4706
iddle2598
iddle2599
iddle4705
iddle4899
iddle4900
iddle1572
iddle1573
iddle2363
ch12lev2sec17
iddle1103
iddle4710
iddle4711
ch12lev2sec18
iddle4839
iddle4840
iddle1970
iddle4986
iddle3743
iddle4704
iddle1034
ch13list01
iddle1365
iddle3840
iddle3088
iddle3089
iddle3159
iddle3160
iddle1754
iddle3324
iddle3325
iddle4715
ch13fn01
ch13list02
ch13lev2sec1
iddle4441
iddle1302
iddle2281
iddle1318
iddle4556
iddle4557
iddle2359
iddle2360
iddle3254
iddle3255
iddle1448
ch13lev2sec2
iddle2361
iddle2365
ch13list03
iddle4953
iddle1812
iddle1813
iddle3695
iddle2834
iddle3039
iddle3040
ch13list04
iddle3099
iddle3110
iddle3210
iddle3286
ch13list05
ch13lev2sec3
iddle4344
iddle1303
iddle3136
iddle3140
iddle5136
iddle3810
iddle3815
iddle2804
iddle1449
ch13fn02
ch13fig01
iddle4847
iddle4848
iddle3052
iddle3055
iddle3058
iddle1587
iddle1588
iddle3098
iddle4947
iddle4948
iddle3808
iddle4952
iddle3838
ch13fn03
iddle4225
iddle1184
iddle1185
iddle1186
ch13sb01
iddle5143
ch13fn04
iddle4129
iddle3605
iddle4570
iddle3670
iddle3671
ch13fn05
iddle1188
ch13fig02
iddle2611
iddle4175
iddle5106
iddle3120
iddle2838
iddle2322
iddle2323
iddle2328
ch13sb02
iddle3084
iddle3104
iddle3115
iddle3138
iddle3139
iddle4685
iddle4906
iddle1834
iddle1835
iddle3521
iddle4984
iddle1368
iddle3137
iddle1478
ch13list06
iddle3813
iddle3814
iddle3836
iddle3837
iddle4157
iddle4069
iddle4070
iddle1187
iddle2543
iddle1997
iddle2833
ch13fn06
iddle3493
iddle4985
iddle3109
ch13fn07
ch13list07
iddle1971
iddle1972
ch13fig03
iddle2920
iddle2329
iddle3072
iddle3091
iddle1228
iddle3132
iddle3261
iddle4982
iddle4983
iddle3385
iddle2167
iddle3520
iddle1122
ch14list01
iddle1232
iddle1244
iddle1245
iddle1262
iddle1263
iddle3795
iddle3796
iddle1873
iddle18731
iddle1874
iddle1875
iddle1876
iddle1877
iddle3133
iddle3141
ch14lev2sec1
iddle4769
ch14fn01
ch14list02
iddle1987
ch14list03
iddle3806
iddle3820
iddle3839
iddle2117
ch14list04
iddle2224
ch14fn02
iddle2312
iddle2313
iddle2314
iddle4130
iddle1229
iddle1230
iddle1278
ch14lev2sec2
ch14lev2sec3
iddle2081
iddle2118
ch14fn03
ch14fn04
iddle1716
ch14fig01
ch14list05
iddle3645
iddle3646
iddle3105
iddle5019
iddle5020
iddle2181
iddle1231
iddle3642
iddle3643
iddle3644
iddle3682
iddle1267
iddle1505
iddle15051
iddle1371
iddle1372
iddle1373
iddle4387
iddle4388
iddle4389
iddle4391
iddle4393
iddle1503
iddle1507
iddle3403
iddle3404
ch14fn05
iddle1581
iddle1613
ch14list06
iddle4594
iddle1879
iddle1711
ch14fn06
iddle1887
iddle1888
iddle1889
iddle2182
ch14lev2sec4
iddle2087
ch14sb01
iddle1509
iddle2326
iddle2327
iddle3602
iddle3603
iddle1499
iddle1500
iddle15001
iddle1501
iddle1512
iddle1513
iddle2623
ch14sb02
iddle3970
ch14lev2sec5
iddle2824
iddle2835
iddle2088
ch14fn07
iddle1901
iddle1902
ch14fn08
iddle3001
iddle4320
iddle4321
iddle4322
iddle4343
iddle3111
iddle1517
ch14fn09
ch14list07
iddle3396
ch14sb03
iddle3758
iddle2508
iddle2509
ch14lev2sec6
iddle3757
iddle3759
ch14fn10
ch14lev2sec7
iddle1514
iddle3950
iddle5132
ch14sb04
iddle3330
iddle4392
iddle3163
iddle3164
iddle2510
iddle4212
ch14sb05
iddle3772
iddle3641
iddle1519
iddle1520
iddle2511
ch14list08
iddle4964
ch14lev2sec8
iddle2637
iddle5140
iddle1202
iddle3686
iddle4430
iddle4431
ch14lev2sec9
iddle1450
iddle1521
iddle1522
ch14list09
iddle3213
iddle3214
iddle2555
iddle1962
iddle3215
ch14lev2sec10
iddle5055
iddle5056
iddle3310
iddle3311
iddle3312
iddle3313
iddle3314
iddle3315
iddle3316
iddle3317
iddle3318
iddle3331
iddle3332
iddle3336
iddle3337
iddle1118
iddle5133
iddle5134
ch14lev2sec11
iddle1362
iddle2397
iddle2447
iddle1496
iddle1988
iddle1506
iddle1508
iddle1510
iddle1511
iddle3321
iddle3322
iddle3328
iddle3329
iddle3335
iddle2201
iddle3397
iddle4436
iddle5139
iddle5151
iddle3510
ch14list10
iddle1900
iddle2938
iddle2942
iddle2017
iddle2037
iddle2038
ch14sb06
iddle1495
iddle1515
iddle1516
iddle1016
iddle1104
iddle1105
iddle1106
iddle1116
iddle2279
iddle2280
iddle3323
iddle3333
iddle3334
ch14fn11
iddle1789
iddle1790
iddle5135
iddle4290
iddle4291
iddle1497
iddle1498
iddle1502
iddle1595
iddle1619
iddle3799
ch14fn12
iddle2825
ch14list11
iddle4045
iddle3068
iddle2506
iddle2507
ch14list12
iddle2101
iddle2102
iddle3346
iddle3347
iddle2798
iddle1117
iddle4506
iddle4507
iddle2403
iddle2404
iddle2405
iddle2406
iddle2407
iddle3041
iddle3706
iddle3707
iddle3760
iddle1600
iddle1601
iddle1703
ch14list13
iddle3077
iddle2174
ch14lev2sec12
iddle1504
ch14list14
iddle1203
iddle3817
iddle5048
iddle5049
iddle5100
iddle5101
iddle4143
ch14fn13
iddle4192
iddle4193
ch14lev2sec13
iddle4278
iddle4279
iddle3049
iddle3050
iddle3063
iddle3064
ch14fn14
iddle4895
iddle4896
iddle2148
iddle4591
ch14list15
iddle5137
iddle5138
iddle5141
ch14lev2sec14
iddle1858
iddle1859
iddle1989
iddle3834
iddle3835
ch14list16
ch14lev2sec15
iddle1698
iddle4088
iddle4089
ch14lev2sec16
iddle4152
iddle2939
ch14fn15
iddle4404
iddle4405
iddle4592
iddle4593
iddle1022
iddle1023
iddle1054
iddle1058
iddle1137
iddle1170
iddle1174
iddle1324
iddle1484
iddle2431
ch15fn01
iddle1755
iddle3811
iddle3812
iddle1800
ch15fn02
iddle1589
iddle1590
iddle2470
iddle1321
ch15lev2sec1
iddle1389
iddle4137
iddle1605
iddle1606
iddle2918
iddle3818
iddle3819
ch15fn03
iddle3400
ch15list01
iddle3024
iddle3087
iddle3121
ch15lev2sec2
iddle3301
iddle3305
iddle3377
iddle3378
iddle3379
iddle2917
iddle3394
iddle3395
ch15fn04
iddle3427
iddle3428
iddle3429
ch15list02
iddle3533
iddle3534
iddle3535
iddle2617
ch15fn05
iddle1139
iddle1143
iddle1169
ch15lev2sec3
iddle4083
iddle4084
iddle4085
iddle2178
iddle3657
iddle3658
iddle3691
iddle1322
iddle2377
iddle3096
iddle1602
iddle1603
iddle2095
iddle1127
iddle1168
iddle1172
iddle1173
iddle1332
ch15lev2sec4
iddle4512
iddle1523
iddle1524
iddle5045
iddle5065
iddle5123
ch15list03
iddle2332
iddle2333
iddle3307
iddle4827
ch15lev2sec5
iddle1140
ch15fig01
ch15fig02
iddle5126
ch15list04
iddle1171
ch15list05
iddle4513
iddle3515
iddle1052
iddle1055
iddle1056
ch15fn06
iddle1596
iddle1323
iddle1325
iddle2853
iddle2854
iddle2096
ch15lev2sec6
iddle2076
iddle2077
iddle1801
iddle2168
iddle3271
iddle3272
iddle4105
iddle4106
iddle3655
iddle3656
ch15lev2sec7
iddle1053
iddle1064
iddle1065
iddle4552
iddle4553
iddle3060
ch15list06
iddle4122
iddle3487
iddle3488
iddle3507
iddle5042
iddle5043
ch15fig03
iddle3165
ch15fig04
ch15fig05
iddle3788
iddle3789
iddle3302
iddle3303
ch15fn07
iddle3816
iddle3508
iddle2100
iddle3659
iddle5047
iddle5067
ch15list07
iddle5125
iddle2992
iddle1157
iddle1176
ch15lev2sec8
iddle1485
ch15list08
iddle2991
iddle1003
ch15lev2sec9
iddle5044
iddle4869
iddle3212
iddle2129
ch15fn08
ch15fn09
iddle2338
iddle2339
iddle4359
iddle1014
iddle1015
iddle1196
ch16lev2sec1
iddle1040
ch16lev2sec2
iddle3823
iddle3824
iddle4973
ch16list01
ch16fig01
ch16fn01
ch16lev2sec3
ch16fn02
iddle2892
iddle2893
iddle2894
iddle2895
iddle2908
iddle2620
ch16sb01
iddle3202
iddle3216
iddle3217
iddle3218
iddle3219
iddle3224
iddle3230
ch16fn03
iddle3444
ch16fn04
iddle5015
iddle5090
iddle5091
ch16fig02
iddle3195
iddle3200
iddle3201
iddle3205
ch16lev2sec4
iddle1119
iddle1120
iddle3222
iddle3223
iddle3263
iddle3264
iddle2155
iddle1675
iddle1763
iddle2775
ch16list02
iddle4135
iddle4224
iddle2612
iddle2613
iddle3203
iddle3204
iddle4160
iddle4233
iddle4234
ch16lev2sec5
iddle2396
ch16list03
iddle3856
iddle3411
iddle3412
iddle2190
iddle3460
iddle3463
iddle5155
iddle2614
ch16sb02
ch16lev2sec6
iddle3417
iddle3418
iddle5102
iddle5103
ch16fn05
ch16lev2sec7
iddle3413
iddle2211
iddle3464
ch16list04
iddle3554
ch16list05
iddle4013
ch16list06
ch16lev2sec8
iddle1047
iddle1048
iddle1049
iddle2739
iddle4547
iddle2949
iddle3553
iddle2529
iddle1990
ch16list07
iddle3735
iddle2699
iddle2743
iddle4562
ch16sb03
iddle2901
ch16fn06
iddle1943
iddle1944
iddle1945
iddle1968
ch16list08
iddle2696
ch16sb04
iddle1969
iddle1093
iddle1098
iddle1099
iddle1100
biblio01_001
iddle2112
biblio01_002
iddle3727
biblio01_003
biblio01_004
iddle2687
iddle2688
iddle2689
iddle2690
iddle2691
iddle2692
iddle2693
iddle2694
iddle2695
iddle2697
biblio01_005
iddle2738
biblio01_006
biblio01_007
biblio01_008
biblio01_009
iddle1456
biblio01_010
iddle4035
iddle2948
iddle1096
biblio01_011
biblio01_012
biblio01_013
iddle2535
biblio01_014
iddle2172
biblio01_015
biblio01_016
biblio01_017
iddle2744
biblio01_018
iddle1828
biblio01_019
iddle1201
biblio01_020
biblio01_021
biblio01_022
biblio01_023
biblio01_024
iddle1960
iddle2536
biblio01_025
biblio01_026
iddle3514
biblio01_027
biblio01_028
iddle2710
biblio01_029
biblio01_030
biblio01_031
biblio01_032
iddle3860
iddle3861
iddle3862
iddle3863
iddle4440
iddle2480
iddle2704
iddle4036
iddle4537
iddle4538
iddle3853
iddle3339
iddle2891
iddle4861
iddle4939