Implementing Multi-threaded CORBA Applications With ACE And TAO Outline .

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Pros OBJ REF Object (Servant) in args operation() out args return IDL SKEL IDL STUBS int main (void) { ORB Manager orb manager (argc, argv); const char *factory name "my quoter factory"; Cons Object Adapter orb- run() ORB CORE The server creates a single Quoter factory and waits in ORB’s event loop Simple to implement and efficient for long-duration requests Client Douglas C. Schmidt Thread-per-Connection Main Program // Create the servant, which registers with rootPOA // and Naming Service implicitly. My Quoter Factory factory (factory name); Excessive overhead for short-duration requests // Block indefinitely waiting for incoming invocations // and dispatch upcalls. // After run() returns, the ORB has shutdown. orb manager.run (); Permits unbounded number of concurrent requests } The ORB’s svc.conf file static Resource Factory "-ORBResources global -ORBReactorType select mt" static Server Strategy Factory "-ORBConcurrency thread-per-connection" UC Irvine 20 CORBA Tutorial Douglas C. Schmidt UC Irvine CORBA Tutorial Thread-per-Connection: Quoter Interface 21 Douglas C. Schmidt Thread-per-Connection Quoter Implementation typedef u long COUNTER; // Maintain request count. // Implementation of multi-threaded Quoter callback invoked by // the CORBA skeleton long My Quoter::get quote (const char *stock name) throw(CORBA::SystemException,Quoter::InvalidStock) { // Increment the request count (beware.). My Quoter::req count ; // Implementation of the Quoter IDL interface class My Quoter : virtual public POA Stock::Quoter, virtual public PortableServer::RefCountServantBase { public: // Constructor. My Quoter (const char *name); // Obtain stock price (beware.). long value Quote Database::instance ()- lookup stock price (stock name); // Returns the current stock value. long get quote (const char *stock name) throw (CORBA::SystemException, Quoter::InvalidStock); private: // Maintain request count. static COUNTER req count ; }; UC Irvine // Skeleton handles exceptions. if (value -1) throw Stock::Invalid Stock (); return value; } 22 UC Irvine 23

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Eliminating Race Conditions Douglas C. Schmidt Basic Synchronization Mechanisms One approach to solve the serialization problem is Problem // SunOS 5.x, implicitly "unlocked". mutex t lock; – The concurrent Quote server contains “race conditions” e.g., Auto-increment of static variable req count is not serialized properly Quote Database also may not be serialized. long My Quoter::get quote (const char *stock name) throw(CORBA::SystemException,Quoter::InvalidStock) { mutex lock (&lock); // Increment the request count. My Quoter::req count ; Forces – Modern shared memory multi-processors use ‘deep caches and weakly ordered‘ memory models – Access to shared data must be protected from corruption // Obtain stock price. long value Quote Database::instance ()- lookup stock price (stock name); if (value -1) // Skeleton handles exceptions. throw Stock::Invalid Stock (); mutex unlock (&lock); return value; Solution – Use synchronization mechanisms } UC Irvine 24 CORBA Tutorial Douglas C. Schmidt UC Irvine CORBA Tutorial Problems Galore! Douglas C. Schmidt C Wrappers for Synchronization To address portability problems, define a C wrapper: Problems with explicit mutex * calls: – Inelegant “Impedance mismatch” with C/C – Obtrusive Must find and lock all uses of lookup stock price and req count – Error-prone C exception handling and multiple method exit points cause subtle problems Global mutexes may not be initialized correctly. – Non-portable Hard-coded to Solaris 2.x – Inefficient e.g., expensive for certain platforms/designs UC Irvine 25 26 class Thread Mutex { public: Thread Mutex (void) { mutex init (&lock , USYNCH THREAD, 0); } Thread Mutex (void) { mutex destroy (&lock ); } int acquire (void) { return mutex lock (&lock ); } int tryacquire (void) { return mutex trylock (&lock); } int release (void) { return mutex unlock (&lock ); } private: mutex t lock ; // SunOS 5.x serialization mechanism. void operator (const Thread Mutex &); Thread Mutex (const Thread Mutex &); }; Note, this mutual exclusion class interface is portable to other OS platforms UC Irvine 27

CORBA Tutorial Douglas C. Schmidt Porting Thread Mutex to Windows NT Using the C wrapper helps improve portability and elegance: Thread Mutex lock; class Thread Mutex { public: Thread Mutex (void) { InitializeCriticalSection (&lock ); } Thread Mutex (void) { DeleteCriticalSection (&lock ); } int acquire (void) { EnterCriticalSection (&lock ); return 0; } int tryacquire (void) { TryEnterCriticalSection (&lock ); return 0; } int release (void) { LeaveCriticalSection (&lock ); return 0; } private: CRITICAL SECTION lock ; // Win32 locking mechanism. // . UC Irvine long My Quoter::get quote (const char *stock name) throw(CORBA::SystemException,Quoter::InvalidStock) { lock.acquire (); My Quoter::req count ; // Increment the request count. // Obtain stock price. long value Quote Database::instance ()- lookup stock price (stock name); if (value -1) // Skeleton handles exceptions. throw Stock::Invalid Stock (); lock.release (); return value; } However, it does not solve the obtrusiveness or error-proneness problems. 28 Douglas C. Schmidt UC Irvine 29 CORBA Tutorial Automated Mutex Acquisition and Release Douglas C. Schmidt Using the Guard Class To ensure mutexes are locked and unlocked, we’ll define a template class that acquires and releases a mutex automatically Using the Guard class helps reduce errors: Thread Mutex lock; long My Quoter::get quote (const char *stock name) throw(CORBA::SystemException,Quoter::InvalidStock) { Guard Thread Mutex mon (lock); My Quoter::req count ; // Increment the request count. template class LOCK class Guard { public: Guard (LOCK &m): lock (m) { lock .acquire (); } Guard (void) { lock .release (); } // . private: LOCK &lock ; } // Obtain stock price. long value Quote Database::instance ()- lookup stock price (stock name); if (value -1) // Skeleton handles exceptions. throw Stock::Invalid Stock (); return value; // Destructor of mon release lock. Guard uses the C idiom whereby a ‘constructor acquires a resourceand the destructor releases the resource‘ UC Irvine Douglas C. Schmidt Using the C Thread Mutex Wrapper Win32 version of Thread Mutex CORBA Tutorial CORBA Tutorial } However, using the Thread Mutex and Guard classes is still overly obtrusive and subtle (may lock too much scope.) 30 UC Irvine 31

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Intent Q: Why is Guard parameterized by the type of LOCK? A: there are many locking mechanisms that benefit from Guard functionality, e.g., – – – – – – Douglas C. Schmidt The Wrapper Facade Pattern OO Design Interlude – ‘Encapsulate low-level, stand-alone functions within type-safe, modular, and portable class interfaces’ Non-recursive vs recursive mutexes Intra-process vs inter-process mutexes Readers/writer mutexes Solaris and System V semaphores File locks Null mutex This pattern resolves the following forces that arises when using native C-level OS APIs 1. ‘How to avoid tedious, error-prone, and non-portable programming of low-level IPC and locking mechanisms‘ 2. ‘How to combine multiple related, but independent, functions into a single cohesive abstraction‘ In ACE, all synchronization classes use the Wrapper Facade and Adapter patterns to provide identical interfaces that facilitate parameterization UC Irvine 32 CORBA Tutorial Douglas C. Schmidt UC Irvine 33 CORBA Tutorial Douglas C. Schmidt Using the Wrapper Facade Pattern for Locking Structure of the Wrapper Facade Pattern 1: method k() client 1: acquire () Wrapper Facade client method 1() . method m() Functions function 1() . function n() UC Irvine Mutex acquire() release() tryacquire() Solaris 2: function k() mutex lock() mutex unlock() mutex trylock() 34 UC Irvine 2: mutex lock() 35

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Using the Adapter Pattern for Locking The following C template class uses the “Decorator” pattern to define a set of atomic operations on a type parameter: LOCK client Guard template class LOCK ACE Thread Mutex, class TYPE u long class ACE Atomic Op { public: ACE Atomic Op (TYPE c 0) { count c; } Guard() Guard() 1: Guard() Mutex Guard Guard() Guard() TYPE operator (void) { Guard LOCK m (lock ); } Mutex acquire() 2: acquire() POSIX Solaris mutex lock() Win32 private: LOCK lock ; TYPE count ; }; EnterCritical Section() UC Irvine 36 CORBA Tutorial Douglas C. Schmidt UC Irvine 37 CORBA Tutorial Using ACE Atomic Op Douglas C. Schmidt Thread-safe Version of Quote Server A few minor changes are made to the class header: req count is now serialized automatically so only minimal scope is locked #if defined (MT SAFE) typedef ACE Atomic Op COUNTER; // Note default parameters. #else typedef ACE Atomic Op ACE Null Mutex COUNTER; #endif /* MT SAFE */ long My Quoter::get quote (const char *stock name) { // Increment the request count by calling // ACE Atomic Op::operator (void) decorator. My Quoter::req count ; In addition, we add a lock, producing: class My Quoter : virtual public POA Stock::Quoter, virtual public PortableServer::RefCountServantBase { // . // Serialize access to database. ACE Thread Mutex lock ; // Obtain stock price via decorator. long value Quote Database::instance ()- lookup stock price (stock name); if (value -1) // Skeleton handles exceptions. throw Stock::Invalid Stock (); return value; // Maintain request count. static COUNTER req count ; }; UC Irvine return count ; operator TYPE () { Guard LOCK m (lock ); return count ; } // Other arithmetic operations omitted. 3: mutex lock() pthread mutex lock() Douglas C. Schmidt Parameterizing Synchonization Via C } 38 UC Irvine 39

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Thread Pool Douglas C. Schmidt TAO’s Thread Pool Concurrency Architecture Pros This approach creates a thread pool to amortize the cost of dynamically creating threads Bounds the number of concurrent requests Client OBJ REF In this scheme, before waiting for input the server code creates the following: IDL STUBS 1. A Quoter Factory (as before) 2. A pool of threads based upon the command line input Object (Servant) in args operation() out args return IDL SKEL Object Adapter orb- run() orb- run() Note the use of the ACE spawn n method for spawning multiple pool threads ORB CORE orb- run() Scales nicely for multi-processor platforms, e.g., permits load balancing Cons Potential for Deadlock UC Irvine 40 CORBA Tutorial Douglas C. Schmidt UC Irvine 41 CORBA Tutorial Thread Pool Main Program Douglas C. Schmidt Thread Pool Main Program (cont’d) const int DEFAULT POOL SIZE 8; int pool size argc 2 ? DEFAULT POOL SIZE : atoi (argv[1]); // Create a thread pool. ACE Thread Manager::instance ()- spawn n (pool size, &run orb, (void *) orb manager.orb (), THR DETACHED THR NEW LWP); int main (int argc, char *argv[]) { try { ORB Manager orb manager (argc, argv); const char *factory name "my quoter factory"; // Block indefinitely waiting for other threads to exit. ACE Thread Manager::instance ()- wait (); // Create the servant, which registers with // the rootPOA and Naming Service implicitly. My Quoter Factory factory (factory name); // After run() returns, the ORB has shutdown. } catch (.) { /* handle exception . */ } // . UC Irvine } 42 UC Irvine 43

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Douglas C. Schmidt Thread Pool Configuration Client/Server Structure QUOTE CLIENT The run orb adapter function : My Quoter : Quoter Proxy void run orb (void *arg) { try { CORBA::ORB ptr orb ACE reinterpret cast (CORBA::ORB ptr, arg); : Quoter Proxy remove() QUOTE SERVER : My Quoter name // Block indefinitely waiting for incoming // invocations and dispatch upcalls. orb- run (); : Quoter Factory Proxy Quoter name // After run() returns, the ORB has shutdown. } catch (.) { /* handle exception . */ } : Quoter Proxy } : My Quoter Factory create object() get quote() value : My Quoter The client works with any server concurrency model The client obtains a Quoter Factory object reference, spawns n threads, and obtains a Quoter object reference per-thread Each thread queries the Quoter 100 times to obtain the value of ACME ORB’s stock main() then waits for threads to terminate The ORB’s svc.conf file static Resource Factory "-ORBReactorType tp" UC Irvine 44 CORBA Tutorial Douglas C. Schmidt UC Irvine CORBA Tutorial Client Code Douglas C. Schmidt Main Client Program Client spawns threads to run the get quotes function and waits for threads to exit The entry point function that does a remote invocation to get a stock quote from the server int main (int argc, char *argv[]) { Options::instance ()- parse args (argc, argv); // This method executes in one or more threads. static void *get quotes (void *arg) { Quoter Factory ptr factory static cast Quoter Factory ptr (arg); try { // Narrow to Quoter Factory interface. Quoter Factory var factory bind service Quoter Factory ("My Quoter Factory", argc, argv); CosLifeCycle::Key key Options::instance ()- key (); Quoter var quoter Stock::Quoter:: narrow(factory- create object (key)); if (!CORBA::is nil (quoter)) { for (int i 0; i 100; i ) { try { long value quoter- get quote ("ACME ORBs"); cout "value " value endl; } catch (.) { /* Handle exception */ } } quoter- remove (); } // Create client threads. ACE Thread Manager::instance ()- spawn n (Options::instance ()- threads (), get quotes, (void *) factory, THR DETACHED THR NEW LWP); // Wait for the client threads to exit ACE Thread Manager::instance ()- wait (); } catch (.) { /* . */ } } } UC Irvine 45 46 UC Irvine 47

CORBA Tutorial Douglas C. Schmidt CORBA Tutorial Obtaining an Object Reference via the Naming Service static CORBA::ORB ptr orb; extern CosNaming::NamingContext ptr name context; TAO supports several threading models template class T typename T:: ptr type /* trait */ bind service (const char *n, int argc, char *argv[]) { CORBA::Object var obj; // "First time" check. if (CORBA::is nil (name context)) { // Get reference to name service. orb CORBA::ORB init (argc, argv, 0); obj orb- resolve initial references ("NameService"); name context CosNaming::NamingContext:: narrow (obj); if (CORBA::is nil (name context)) return 0; } CosNaming::Name svc name; svc name.length (1); svc name[0].id n; svc name[0].kind "object impl"; // Find object reference in the name service. obj name context- resolve (svc name); // Narrow to the T interface and away we go! return T:: narrow (obj); } UC Irvine Douglas C. Schmidt Concluding Remarks – Performance may determine model choice ACE provides key building blocks for simplifying concurrent application code – www.cs.wustl.edu/ schmidt/ACE.html More information on CORBA can be obtained at – www.cs.wustl.edu/ schmidt/corba.html C Report columns written with Steve Vinoski – www.cs.wustl.edu/ schmidt/report-doc.html 48 UC Irvine 49

ACE provides portable C threading and synchronization wrappers ACE classes we'll examine include: - Thread Management! ACE Thread Manager encapsulates threads - Synchronization! ACE Thread Mutexand ACE RW Mutex encapsulates mutexes! ACE Atomic Op atomically perform arithmetic operations! ACE Guard automatically acquire/release locks .