ACE Tutorial 006
Creating a thread-per-connection server

client_handler.cpp exposes all the things I've been hinting at. Pay special attention to the decision made in open() as well as the bit of cleverness in svc().

    // page05.html,v 1.14 2000/03/19 20:09:23 jcej Exp

    /* In client_handler.h I alluded to the fact that we'll mess around
       with a Client_Acceptor pointer.  To do so, we need the
       Client_Acceptor object declaration.

       We know that including client_handler.h is redundant because
       client_acceptor.h includes it.  Still, the sentry prevents
       double-inclusion from causing problems and it's sometimes good to
       be explicit about what we're using.

       On the other hand, we don't directly include any ACE header files
       here.  */
    #include "client_acceptor.h"
    #include "client_handler.h"

    /* Our constructor doesn't do anything.  That's generally a good idea.
       Unless you want to start throwing exceptions, there isn't a really
       good way to indicate that a constructor has failed.  If I had my
       way, I'd have a boolean return code from it that would cause new to
       return 0 if I failed.  Oh well...  */
    Client_Handler::Client_Handler (void)
    {
    }

    /* Our destructor doesn't do anything either.  That is also by design.
       Remember, we really want folks to use destroy() to get rid of us.
       If that's so, then there's nothing left to do when the destructor
       gets invoked.  */
    Client_Handler::~Client_Handler (void)
    {
    }

    /* The much talked about destroy() method!  The reason I keep going on
       about this is because it's just a Bad Idea (TM) to do real work
       inside of a destructor.  Although this method is void, it really
       should return int so that it can tell the caller there was a
       problem.  Even as void you could at least throw an exception which
       you would never want to do in a destructor.  */
    void
    Client_Handler::destroy (void)
    {
      /* Tell the reactor to forget all about us.  Notice that we use the
        same args here that we use in the open() method to register
        ourselves.  In addition, we use the DONT_CALL flag to prevent
        handle_close() being called.  Since we likely got here due to
        handle_close(), that could cause a bit of nasty recursion!  */
      this->reactor ()->remove_handler (this,
                                        ACE_Event_Handler::READ_MASK
                                        | ACE_Event_Handler::DONT_CALL);

      /* This is how we're able to tell folks not to use delete.  By
        deleting our own instance, we take care of memory leaks after
        ensuring that the object is shut down correctly.  */
      delete this;
    }

    /* As mentioned before, the open() method is called by the
       Client_Acceptor when a new client connection has been accepted.
       The Client_Acceptor instance pointer is cast to a void* and given
       to us here.  We'll use that to avoid some global data...  */
    int
    Client_Handler::open (void *void_acceptor)
    {
      /* We need this to store the address of the client that we are now
         connected to.  We'll use it later to display a debug message.  */
      ACE_INET_Addr addr;

      /* Our ACE_Svc_Handler baseclass gives us the peer() method as a way
        to access our underlying ACE_SOCK_Stream.  On that object, we can
        invoke the get_remote_addr() method to get get an ACE_INET_Addr
        having our client's address information. As with most ACE methods,
        we'll get back (and return) a -1 if there was any kind of error.
        Once we have the ACE_INET_Addr, we can query it to find out the
        client's host name, TCP/IP address, TCP/IP port value and so
        forth.  One word of warning: the get_host_name() method of
        ACE_INET_Addr may return you an empty string if your name server
        can't resolve it.  On the other hand, get_host_addr() will always
        give you the dotted-decimal string representing the TCP/IP
        address.  */
      if (this->peer ().get_remote_addr (addr) == -1)
        return -1;

      /* Convert the void* to a Client_Acceptor*.  You should probably use
        those fancy ACE_*_cast macros but I can never remember how/when to
        do so.  Since you can cast just about anything around a void*
        without compiler warnings be very sure of what you're doing when
        you do this kind of thing.  That's where the new-style cast
        operators can save you.  */
      Client_Acceptor *acceptor = (Client_Acceptor *) void_acceptor;

      /* Our Client_Acceptor is constructed with a concurrency strategy.
        Here, we go back to it to find out what that strategy was.  If
        thread-per-connection was selected then we simply activate a
        thread for ourselves and exit.  Our svc() method will then begin
        executing in that thread.

        If we are told to use the single-threaded strategy, there is no
        difference between this and the Tutorial 5 implementation.

        Note that if we're in thread-per-connection mode, open() is exited
        at this point.  Furthermore, thread-per-connection mode does not
        use the reactor which means that handle_input() and it's fellows
        are not invoked.  */
      if (acceptor->thread_per_connection ())
        return this->activate (THR_DETACHED);

      // ************************************************************************
      // From here on, we're doing the traditional reactor thing.  If
      // you're operating in thread-per-connection mode, this code does
      // not apply.
      // ************************************************************************

      /* Our reactor reference will be set when we register ourselves but
        I decided to go ahead and set it here.  No good reason really...  */
      this->reactor (acceptor->reactor ());

      /* If we managed to get the client's address then we're connected to
        a real and valid client.  I suppose that in some cases, the client
        may connect and disconnect so quickly that it is invalid by the
        time we get here. In any case, the test above should always be
        done to ensure that the connection is worth keeping.

        Now, regiser ourselves with a reactor and tell that reactor that
        we want to be notified when there is something to read.  Remember,
        we took our reactor value from the acceptor which created us in
        the first place.  Since we're exploring a single-threaded
        implementation, this is the correct thing to do.  */
      if (this->reactor ()->register_handler (this,
                                              ACE_Event_Handler::READ_MASK) == -1)
        ACE_ERROR_RETURN ((LM_ERROR,
                           "(%P|%t) can't register with reactor\n"),
                          -1);

      /* Here, we use the ACE_INET_Addr object to print a message with the
        name of the client we're connected to.  Again, it is possible that
        you'll get an empty string for the host name if your DNS isn't
        configured correctly or if there is some other reason that a
        TCP/IP addreess cannot be converted into a host name.  */
      ACE_DEBUG ((LM_DEBUG,
                  "(%P|%t) connected with %s\n", addr.get_host_name ()));

      /* Always return zero on success.  */
      return 0;
    }

    /* As mentioned in the header, the typical way to close an object in a
       threaded context is to invoke it's close() method.  Since we
       already have a handle_close() method built to cleanup after us,
       we'll just forward the request on to that object.  */
    int
    Client_Handler::close(u_long flags)
    {
      ACE_UNUSED_ARG (flags);

      /* We use the destroy() method to clean up after ourselves.  That
        will take care of removing us from the reactor and then freeing
        our memory.  */
      this->destroy ();

      /* Don't forward the close() to the baseclass!  handle_close() above
        has already taken care of delete'ing.  Forwarding close() would
        cause that to happen again and things would get really ugly at
        that point!  */
      return 0;
    }

    /* In the open() method, we registered with the reactor and requested
       to be notified when there is data to be read.  When the reactor
       sees that activity it will invoke this handle_input() method on us.
       As I mentioned, the _handle parameter isn't useful to us but it
       narrows the list of methods the reactor has to worry about and the
       list of possible virtual functions we would have to override.

       Again, this is not used if we're in thread-per-connection mode.  */
    int
    Client_Handler::handle_input (ACE_HANDLE handle)
    {
      /* Some compilers don't like it when you fail to use a parameter.
        This macro will keep 'em quiet for you.  */
      ACE_UNUSED_ARG (handle);

      /* Now, we create and initialize a buffer for receiving the data.
        Since this is just a simple test app, we'll use a small buffer
        size.  */
      char buf[BUFSIZ];

      /* Invoke the process() method with a pointer to our data area.
        We'll let that method worry about interfacing with the data.  You
        might choose to go ahead and read the data and then pass the
        result to process().  However, application logic may require that
        you read a few bytes to determine what else to read...  It's best
        if we push that all into the application-logic level.  */
      return this->process (buf, sizeof (buf));
    }

    /* If we return -1 out of handle_input() or if the reactor sees other
       problems with us then handle_close() will be called.  The reactor
       framework will take care of removing us (due to the -1), so we
       don't need to use the destroy() method.  Instead, we just delete
       ourselves directly.  */
    int
    Client_Handler::handle_close (ACE_HANDLE handle,
                                  ACE_Reactor_Mask mask)
    {
      ACE_UNUSED_ARG (handle);
      ACE_UNUSED_ARG (mask);

      this->destroy ();
      return 0;
    }

    /* The ACE_Svc_Handler<> is ultimately derived from ACE_Task<>.  If
       you want to create a multi-threaded application, these are your
       tools!  Simply override the svc() method in your derivative and
       arrange for your activate() method to be called.  The svc() method
       then executes in the new thread.

       Of course, this is only valid if we're in thread-per-connection
       mode.  If we're using the reactor model, then svc() never comes
       into play.  */
    int
    Client_Handler::svc(void)
    {
      /* Like handle_input(), we create a buffer for loading the data.
        Doing so in handle_input() doesn't help any but there is a small
        performance increase by doing this here: the buffer is created
        once when the thread is created instead of for each invocation of
        process().  */
      char buf[BUFSIZ];

      // Forever...
      while( 1 )
        {
          /* Invoke the process() method to read and process the data.
            This is exactly the way it is used by handle_input().  That's
            the reason I created process() in the first place: so that it
            can be used in either concurrency strategy.  Since process()
            has all of our application-level logic, it's nice that it
            doesn't have to change when we decide to go multi-threaded.

            Notice that since the recv() method call in process() blocks until
            there is data ready, this thread doesn't consume any CPU time until
            there is actually data sent from the client.  */
          if (this->process(buf, sizeof (buf)) == -1)
            return -1;
        }

      return 0;
    }

    /* And, at last, we get to the application-logic level.  Out of
       everything we've done so far, this is the only thing that really
       has anything to do with what your application will do.  In this
       method we will read and process the client's data.  In a real
       appliation, you will probably have a bit more in main() to deal
       with command line options but after that point, all of the action
       takes place here.  */
    int
    Client_Handler::process (char *rdbuf,
                             int rdbuf_len)
    {
      /* Using the buffer provided for us, we read the data from the
        client. If there is a read error (eg -- recv() returns -1) then
        it's a pretty good bet that the connection is gone.  Likewise, if
        we read zero bytes then something wrong has happened.  The reactor
        wouldn't have called us if there wasn't some kind of read activity
        but there wouldn't be activity if there were no bytes to read...

        On the other hand, if we got some data then we can display it in a
        debug message for everyone to see.  */
      ssize_t bytes_read;
      switch ( (bytes_read = this->peer ().recv (rdbuf, rdbuf_len)) )
        {
        case -1:
          ACE_ERROR_RETURN ((LM_ERROR,
                             "(%P|%t) %p bad read\n",
                             "client"),
                            -1);
        case 0:
          ACE_ERROR_RETURN ((LM_ERROR,
                             "(%P|%t) closing daemon (fd = %d)\n",
                             this->get_handle ()),
                            -1);
        default:
          rdbuf[bytes_read] = 0;
          ACE_DEBUG ((LM_DEBUG,
                      "(%P|%t) from client: %s",
                      rdbuf));
        }

      return 0;
    }

Did you notice the THR_DETACHED flag on the call to activate()? Threads, like any system resource, are a limited resource. Unless we intend to join() or wait() for the new thread later, we want use THR_DETACHED so that we don't cause a leak. In fact, in most cases, you'll want to specify THR_DETACHED because it's just easier.

Another handy flag for use with activate() is THR_NEW_LWP. That's short for Light Weight Process. If you've got a multiprocessor, this flag will allocate a new schedulable process and decrease the odds of your threads all fighting for the same process. Of course, if you have a uni-processor, it will neither help nor hurt. Since I developed these on a uni-processor, I've been a bit inconsistent in the use of THR_NEW_LWP.

Well, that's it! After all the talk & the hype, you would have expected it to be more difficult to create a multi-threaded server. Surprise! It really is that easy. You still have to handle contention issues which we haven't addressed here and that is a rather nasty topic. Still, for the simple case, this is all you have to do.

The next page is the last for this tutorial. Head on over there & we'll round up the file list one last time.

[Tutorial Index] [Continue This Tutorial]