IOCP Introduction

An introduction to the concepts of I/O Completion ports by analogy for the non technical.

Let's pretend you work for a large corporation in the container shipping business, IOCP Inc. The company divides up its operations by floor. Each floor is completely separate and communication between floors is not permitted.

As a result, you have no idea what goes on in most of the building other than the ground floor is the main() floor in which workers like you are hired, the first floor is sales which generates the work, and floors 2 and 3 are where the work is managed by a manager like you.

There is only one manager per floor and you have all the resources of that floor available to you.

This corporation is only big enough to afford two floors (processors) with managers, but they could move to a new building with more at any time.

Operational Layout

Ground Floor - Main Thread :

First Floor - ListenThread :
Second Floor - Worker Thread:

Third Floor - Worker Thread:

This company has designed a system for dealing with work flow that is centered around an "inbox" (Queue). New work is added to the bottom, managers take work from the top (First in First out).

Managers are very conscientious and stand by the "inbox" waiting for work. However, there is a pecking order. It turns out you are first in line. You perform ALL the work until you are too busy to take on any more. Only then does the next manager in line (from another floor) get assigned work.

This central "inbox" where all the work is stacked is technically called the "completion portal",  because it is the central location where job completion notices arrive.

Your Job as floor manager, is to know what to do when the job completion notice arrives and is handed to you. You may schedule more work yourself or read from disk or send something over the internet.

Sales (on the first floor) gets business from clients. They accept the business and issue a work order for the shipment to begin. That is ALL they do. After that its up to the managers to manage the work flow (scheduling of more jobs, processing etc) in order to complete the project!

When a client contacts the company to begin a shipment (Client connection), sales accepts the job and issues the first work order. Then they go back to "looking busy" and playing whiffle ball. Their work order is very simple. It tells the port to unload any containers shipped from the client. When this order completes, you are notified (with a notification in the central "inbox") that the order has been completed.

In fact the first thing you know about this job is when the first notification arrives. It contains limited information, the address of the shipper and how many containers have been offloaded. This is where your work begins.

What you are NOT told is how big the shipment is, meaning how many containers (bytes) were shipped or what to do with them. That can only be determined once they have all arrived.

Unfortunately there is a complication. The high seas are a treacherous highway, fraught with all kinds of unexpected danger. Containers are frequently lost and must be resent causing most shipments to arrive in stages. (shipments are guaranteed by the shipping company to arrive with containers in the same order they were sent, but when they arrive is not known)

Meanwhile down at the port, when a ship arrives the containers are offloaded, you are immediately notified. As far as the port is concerned, the job has completed. The sales department told them to unload containers from the client and they did. They are now sitting on the dock taking up space.

Your problem is that you must decide if they are all present or if there are more to come on another ship. If they are all there, then you then need to figure out what to do with them and issue and order to get it done. But i'm getting ahead of myself.

As you have probably figured out, you may receive a job completion notification when only one container is sitting on the dock. The workers at the dock have no way of knowing when the entire shipment has arrived, they just notify you when they have unloaded a ship at your port. This is a problem you must deal with.

You must determine if all the containers have been offloaded onto the dock by opening them.  If they are not all present on the dock, you must issue another order to the port to unload any containers shipped from the client (and move on to something else until they arrive). Once again, you will be notified when more have arrived with the usual job completion notification sent to the "inbox" (completion portal).

This process is repeated until at some point, you will have enough containers to figure out how many are in the shipment. If the number of containers on the dock equals the number in the shipment, you can stop issuing receive orders to the dock workers and process the entire shipment by issuing a work order to some processing company. You must decide what needs to be done and issue the orders to do it.

You may decide that all the containers need to be emptied and the contents stored at the company warehouse (hard drive) or that they were empty and need to be filled and returned. Either way you issue an order to do these things, and you will be notified via the completion port when each part is done.

What you DO NOT do, is get in your car, drive to the warehouse and start digging out items and packing containers. If you do that, no other completion notifications get processed by you while you are away and the company has to wake up your substitute, calls him in, and have him take over while you are tied up.

The company does not like to do this because your office has to be emptied and your replacements office furniture and personal items all need to be installed before he can do any work (context switch). This is expensive.

Let's assume that you need to return many containers filled with whatever the client requested. You issue the request to the warehouse to fill the containers and get them to the port ready for shipping. This may take some time, although you have no way of knowing how long, but you do not care. You continue processing other notifications (or just wait if there is nothing to do)  until you are notified that the shipment is ready.

When you are notified that the containers are on the dock ready to be sent, you can issue the order to the dock workers to load them onto the ship and send them back to the client.This may also take a while, but you do not need to be concerned with any of the details. You will be notified when the job is complete.

When that last completion notification arrives, the project is essentially done. All you need do to cleanup is release the port.

Keep in mind that in between each of these notifications you may have processed many other shipments.

If you get more notifications than you can possibly handle each day, then its probably time the company expanded and added a few more floors with all the money they should be making so clients do not have to wait.

Read more about design and implementation here.