Changeset b14ad0728b8f091a7d46d3ae5b151e3dbf3d049a in xas-spooler


Ignore:
Timestamp:
06/29/15 10:23:22 (4 years ago)
Author:
Kevin L. Esteb <kevin@…>
Branches:
master
Children:
6872e8ca659e79210354178cc7e8fea7569dd1fe
Parents:
d06781b7c5a08d0a9b48202eae5e381d6f2ef2c1
git-author:
Kevin L. Esteb <kevin@…> (06/29/15 10:23:22)
git-committer:
Kevin L. Esteb <kevin@…> (06/29/15 10:23:22)
Message:

Fixing documentation

Files:
5 edited

Legend:

Unmodified
Added
Removed
  • README.md

    rbc98338cabfed7789d32f588af9e577d150d2be9 rb14ad0728b8f091a7d46d3ae5b151e3dbf3d049a  
    11XAS Spooler - A spooler for the XAS Environment 
    2 ========================================================= 
     2=============================================== 
    33 
    44XAS is a set of modules, procedures and practices to help write 
  • trunk/lib/XAS/Msgs/Spooler.msg

    re9ef167d6897d11e47e293782c59f99538520e35 rb14ad0728b8f091a7d46d3ae5b151e3dbf3d049a  
    88 
    99[messages] 
    10 connected  = %s: connected to %s on %s 
    11 unlinking  = %s: unlinking %s 
    12 found      = %s: found spool file %s 
    13 files      = %s: %s files waiting 
     10spooler_connected  = %s: connected to %s on %s 
     11spooler_unlinking  = %s: unlinking %s 
     12spooler_found      = %s: found spool file %s 
     13spooler_files      = %s: %s files waiting 
     14spooler_unexpected = %s: unexpected error: %s 
     15 
  • trunk/lib/XAS/Spooler.pm

    r6ea4a2a77d754ccad853ab517365a6e395f8427a rb14ad0728b8f091a7d46d3ae5b151e3dbf3d049a  
    1919sent to queues on a STOMP based message queue server.  
    2020 
    21 These modules only support the self generated Alerts and the direct logging  
    22 for the logstash logging option.  
    23  
    24 =head1 SUPPORT 
    25  
    26 You can find documentation for this module with the perldoc command. 
    27  
    28     perldoc XAS::Spooler 
    29  
    3021=head1 SEE ALSO 
    3122 
     
    3324 
    3425=item L<XAS|XAS> 
     26 
     27=item L<XAS::Apps::Spooler::Process|XAS::Apps::Spooler::Process> 
     28 
     29=item L<XAS::Spooler::Connector|XAS::Spooler::Connector> 
     30 
     31=item L<XAS::Spooler::Processor|XAS::Spooler::Processor> 
    3532 
    3633=back 
  • trunk/lib/XAS/Spooler/Connector.pm

    r6ea4a2a77d754ccad853ab517365a6e395f8427a rb14ad0728b8f091a7d46d3ae5b151e3dbf3d049a  
    1717  vars => { 
    1818    PARAMS => { 
    19       -hostname  => 1, 
     19      -hostname  => { optional => 1, default => undef }, 
    2020    } 
    2121  } 
     
    7171 
    7272sub send_packet { 
    73     my ($self, $palias, $type, $queue, $data, $file) = @_[OBJECT,ARG0..ARG5]; 
     73    my ($self, $palias, $type, $queue, $data, $file) = @_[OBJECT,ARG0..ARG4]; 
    7474 
    7575    my $alias = $self->alias; 
     
    139139 
    140140    my $self = $class->SUPER::init(@_); 
     141 
     142    unless (defined($self->{hostname})) { 
     143 
     144        $self->{hostname} = $self->env->host; 
     145 
     146    } 
    141147 
    142148    $self->{events} = XAS::Lib::POE::PubSub->new(); 
     
    169175=head1 DESCRIPTION 
    170176 
    171 This module use to connect to a message queue server for spoolers. It provides 
    172 the necessary events and methods so the Factory can do its job. 
    173  
    174 =head1 PUBLIC METHODS 
     177This module connects to a message queue server for spoolers. All of the spool 
     178processors funnel messages thru this module. If the connection is lost to 
     179the server, it signals the processor to stop processing until it is able to 
     180reconnect to the server. 
     181 
     182=head1 METHODS 
    175183 
    176184=head2 new 
    177185 
    178 This method creates the initial session, setups the scheduling for  
    179 gather_data() and initializes JSON processing. It takes the following 
    180 configuration items: 
     186This method inherits from L<XAS::Lib::Stomp::POE::Client|XAS::Lib::Stomp::POE::Client> 
     187and takes these additional parameters: 
    181188 
    182189=over 
    183190 
    184 =item B<-processor> 
    185  
    186 A pointer to the ProcessFactory object. 
    187  
    188 =item B<-queue> 
    189  
    190 The name of the queue to send messages to on the message queue server. 
    191  
    192191=item B<-hostname> 
    193192 
    194 The name of the host that this is running on. 
     193An optional name for the host that is processing these spool files. 
    195194 
    196195=back 
     
    198197=head1 PUBLIC EVENTS 
    199198 
    200 =head2 connection_down 
    201  
    202 This event signal that the connection had been dropped, we are just stopping  
    203 the collection of data. This is done by notifing the ProcessFactory that 
    204 data collection should stop. 
    205  
    206 =head2 send_packet 
    207  
    208 This event will format the data to be sent to the message queue server. 
     199=head2 connection_down(OBJECT) 
     200 
     201This event broadcasts that the connection had been dropped. 
     202 
     203=over 4 
     204 
     205=item B<OBJECT> 
     206 
     207The handle for the current self. 
     208 
     209=back 
     210 
     211=head2 connection_up(OBJECT) 
     212 
     213This event broadcasts when the connection is established. 
     214 
     215=over 4 
     216 
     217=item B<OBJECT> 
     218 
     219The handle for the current self. 
     220 
     221=back 
     222 
     223=head2 send_packet(OBJECT,ARG0, ARG1, ARG2, ARG3, ARG4) 
     224 
     225Process the data received from the processors. This processing includes 
     226creating the standard message header, decoding the data and creating a 
     227serialized message using JSON. This message is then sent to message queue 
     228server. 
     229 
     230=over 4 
     231 
     232=item B<OBJECT> 
     233 
     234The handle for the current self. 
     235 
     236=item B<ARG0> 
     237 
     238The alias of the processor. 
     239 
     240=item B<ARG1> 
     241 
     242The type of data. 
     243 
     244=item B<ARG2> 
     245 
     246The queue to send the message too. 
     247 
     248=item B<ARG3> 
     249 
     250The actual data to process. This is usually a JSON formated string. 
     251 
     252=item B<ARG4> 
     253 
     254The full qualified name of the file that was processed. This, along with 
     255the processor alias, is used for the STOMP receipt. 
     256 
     257=back 
    209258 
    210259=head1 SEE ALSO 
     
    212261=over 4 
    213262 
     263=item L<XAS::Spooler|XAS::Spooler> 
     264 
    214265=item L<XAS|XAS> 
    215266 
    216 =item L<XAS::Spooler|XAS::Spooler> 
    217  
    218267=back 
    219268 
     
    224273=head1 COPYRIGHT AND LICENSE 
    225274 
    226 Copyright (C) 2014 by Kevin L. Esteb 
    227  
    228 This library is free software; you can redistribute it and/or modify 
    229 it under the same terms as Perl itself, either Perl version 5.8.8 or, 
    230 at your option, any later version of Perl 5 you may have available. 
     275Copyright (c) 2014 Kevin L. Esteb 
     276 
     277This is free software; you can redistribute it and/or modify it under 
     278the terms of the Artistic License 2.0. For details, see the full text 
     279of the license at http://www.perlfoundation.org/artistic_license_2_0. 
    231280 
    232281=cut 
  • trunk/lib/XAS/Spooler/Processor.pm

    r6ea4a2a77d754ccad853ab517365a6e395f8427a rb14ad0728b8f091a7d46d3ae5b151e3dbf3d049a  
    133133    $self->log->debug("$alias: entering unlink_file()"); 
    134134 
    135     $self->log->info_msg('unlinking', $alias, $file); 
    136  
    137     $self->spooler->delete($file); 
     135    try { 
     136 
     137        $self->log->info_msg('spooler_unlinking', $alias, $file); 
     138        $self->spooler->delete($file); 
     139         
     140    } catch { 
     141 
     142        my $ex = $_; 
     143 
     144        $self->log->error_msg('spooler_unexpected', $alias, $ex); 
     145 
     146    }; 
     147 
    138148    $poe_kernel->post($alias, 'process_files'); 
    139149 
     
    154164    $self->{count} -= 1; 
    155165    $self->{count} = 1 if ($self->{count} < 0); 
    156      
     166 
    157167    $self->log->debug("$alias: task count: " . $self->{count}); 
    158168 
     
    165175                if (my $data = $self->spooler->read($file)) { 
    166176 
    167                     $self->log->info_msg('found', $alias, $file->path); 
     177                    $self->log->info_msg('spooler_found', $alias, $file->path); 
    168178                    $poe_kernel->post($connector, 'send_packet', $alias, $type, $queue, $data, $file->path); 
    169179 
     
    427437=head1 DESCRIPTION 
    428438 
    429 This module scans a spool directory. When any files are found the are  
    430 processed and sent to the Connector. 
    431  
    432 =head1 EVENTS 
    433  
    434 This module responds to the following POE events. 
    435  
    436 =head2 startup 
    437  
    438 Fires the start_scan event. 
    439  
    440 =head2 start_scan 
    441  
    442 Schedules the scanning process. 
    443  
    444 =head2 stop_scan 
    445  
    446 Stops the scanning process. 
    447  
    448 =head2 scan 
    449  
    450 Performs the scanning process and dispatchs any packets to the Connectors  
    451 'send_packet' event. 
    452  
    453 =head2 unlink_file 
     439This module scans a spool directory and process any files that are found.  
     440Discretionary locking is used to coordinate access to the directory.  
     441Scheduling uses cron semantics. 
     442 
     443=head1 METHODS 
     444 
     445=head2 new 
     446 
     447This module inherits from L<XAS::Lib::POE::Service|XAS::Lib::POE::Service> and  
     448takes these additional parameters: 
     449 
     450=over 4 
     451 
     452=item B<-queue> 
     453 
     454The queue to send the messages too. 
     455 
     456=item B<-connector> 
     457 
     458The alias of the connector session. 
     459 
     460=item B<-directory> 
     461 
     462The directory to scan. The can be relative to $XAS_SPOOL or a fully 
     463qualified path. 
     464 
     465=item B<-packet_type> 
     466 
     467The name of the message type.  
     468 
     469=item B<-tasks> 
     470 
     471The number of sessions processing the internal queue of messages, defaults to 1 session. 
     472 
     473=item B<-schedule> 
     474 
     475The schedule used to process items in the spool directory. This uses cron 
     476semantics and defaults to this: '*/1 * * * *' 
     477 
     478=back 
     479 
     480=head1 PUBLIC EVENTS 
     481 
     482=head2 scan(OBJECT) 
     483 
     484Starts the directory scan process. 
     485 
     486=over 4 
     487 
     488=item B<OBJECT> 
     489 
     490The handle for the current self. 
     491 
     492=back 
     493 
     494=head2 scan_dir(OBJECT) 
     495 
     496Reads one file from the directory and queues it to the internal processing 
     497queue. 
     498 
     499=over 4 
     500 
     501=item B<OBJECT> 
     502 
     503The handle for the current self. 
     504 
     505=back 
     506 
     507=head2 scan_dir_stop(OBJECT) 
     508 
     509Stops the directory scan process. 
     510 
     511=over 4 
     512 
     513=item B<OBJECT> 
     514 
     515The handle for the current self. 
     516 
     517=back 
     518 
     519=head2 unlink_file(OBJECT, ARG0) 
    454520 
    455521Removes the unneeded file from the directory. 
    456522 
     523=over 4 
     524 
     525=item B<OBJECT> 
     526 
     527The handle for the current self. 
     528 
     529=item B<ARG0> 
     530 
     531The name of the file to remove. 
     532 
     533=back 
     534 
     535=head2 process_files(OBJECT) 
     536 
     537Removes one file from the internal queue and sends the file contents to the  
     538connector for processing. 
     539 
     540=over 4 
     541 
     542=item B<OBJECT> 
     543 
     544The handle for the current self. 
     545 
     546=back 
     547 
     548=head2 pause_processing(OBJECT) 
     549 
     550Pauses proccessing because the connection to the message queue server 
     551is down. 
     552 
     553=over 4 
     554 
     555=item B<OBJECT> 
     556 
     557The handle for the current self. 
     558 
     559=back 
     560 
     561=head2 resume_processing(OBJECT) 
     562 
     563Resumes processing as the connection to the message queue server 
     564is active. 
     565 
     566=over 4 
     567 
     568=item B<OBJECT> 
     569 
     570The handle for the current self. 
     571 
     572=back 
     573 
    457574=head1 SEE ALSO 
    458575 
    459576=over 4 
    460577 
     578=item L<XAS::Spooler|XAS::Spooler> 
     579 
    461580=item L<XAS|XAS> 
    462581 
    463 =item L<XAS::Spooler|XAS::Spooler> 
    464  
    465582=back 
    466583 
     
    471588=head1 LICENSE AND COPYRIGHT 
    472589 
    473 Copyright 2014 Kevin L. Esteb. 
    474  
    475 This program is free software; you can redistribute it and/or modify it 
    476 under the terms of either: the GNU General Public License as published 
    477 by the Free Software Foundation; or the Artistic License. 
    478  
    479 See http://dev.perl.org/licenses/ for more information. 
     590Copyright (c) 2014 Kevin L. Esteb 
     591 
     592This is free software; you can redistribute it and/or modify it under 
     593the terms of the Artistic License 2.0. For details, see the full text 
     594of the license at http://www.perlfoundation.org/artistic_license_2_0. 
    480595 
    481596=cut 
Note: See TracChangeset for help on using the changeset viewer.