Data Flow

Generation of events

This section describes the steps happening in the process in order to generate monitoring events.
  1. New process is started on the system:
    1. Windows automatically loads "abpa_loader.dll";
    2. "abpa_loader" decides to load "ABPAEngine.dll" (please see Configuration page for details).
  2. Hooks are installed on relevant system calls. For example, if configured to collect File events, a hook will be installed on CreateFile Win32 API function (among other file-related functions).
  3. When the application initiates a call to CreateFile function, actually our HookCreateFile function will be called instead:
    1. It will call the original Win32 function, and will store the returned value (HANDLE to a file);
    2. New FileEvent object will be allocated, and initialized with operation type (CREATE), file name and returned HANDLE;
    3. The evebt object will be sent to Post Office (see the next section);
    4. The value of the HANDLE returned by original Windows CreateFile function will be returned to the callee.

Post Office

All events generated by hook functions (being created in different threads) are eventually processed by dedicated PostOffice thread, which is created as part of initialization of "ABPAEngine.dll". This section describes the mechanism for transferring events from original threads to PostOffice thread.
  • PostOffice manages a separate queue of events for each thread. The queue is implemented in a way that allows no-lock add/fetch access to its elements (described below).

In User Thread

  1. Once a new event is ready to be sent (e.g. FileEvent from our example), PostOffice first finds the queue associated with current thread:
    1. Pointer to thread's queue is stored in Thread Local Storage (TLS) cell, so there's really no need to "find" it;
    2. When first event is being sent by current thread, TLS returns a NULL pointer, and PostOffice would allocate a new queue to be used by the thread.
  2. Event is added to the queue,
    1. Counter of pending events is increased;
    2. When the counter reaches certain value, a special per-queue Condition (Windows "Event" object) will be signaled.

In PostOffice Thread

This thread runs in infinite loop (until stopped). On each iteration it performs the following steps:
  1. Go over the list of all per-thread event queues, and store a handle to each per-queue Event.
  2. Wait until:
    1. one of Conditions is signaled (relevant event queue reached certain number of unprocessed events);
    2. OR timeout of 1/4 second.
  3. Go over all the per-thread queues:
    1. Send each event object to the output file (see the next section);
    2. Delete the object;
    3. Decrease the counter of pending events in the queue.

No-lock Queue

The queue is implemented using STL List, and it keept 2 iterators: the first "write" iterator to the last element in the list and second "read" iterator pointing to the next element to be consumed.
  • All new events are added to the end of the list, and "write" iterator is advanced.
  • While "read" iterator is not equal to "write" iterator (meaning that it points on some earlier element in the list), we can consume the object pointed by "read" and advance "read" iterator to the next element.
  • This way we never consume the last element during the life-time of the thread. When the thread is being terminarted (or the whole process exits), all "orphan" events are collected and processed by PostOffice

Saving to file

Each monitoring Event implements ISerializable interface, defining functions to save the event to "std::ostream", and read the event from "std::istream" (in run-time we only save events, they may be read later by analysis application). This save function is utilized in order to save the event in binary format in the output file.

Last edited Aug 21, 2008 at 8:49 AM by migo, version 1


No comments yet.