I have been working with Mojolicious for a few weeks now. My main goal is to create a ReST API to send and receive DNS queries: a useful mechanism for my PhD thesis (a topic for a future post).

In this post you are going to create a Mojolicious + Minion almost Rest API full example to enqueue jobs and learn how to use Minion job/worker command to manage yours queues. I assume that you already have installed Mojolicious and Minion.

First things first – Environment

My setup has the following basic tools under perlbrew:

  1. Perl version: v5.24.0
  2. Mojolicious: 7.31
  3. Minion: 6.05
  4. Postgres: 9.6 (Minions need that, it does not work with MySQL)
  5. PerlBrew!
  6. Debian Stable

Main Code – Routes

The following code denotes a WebServer which receives an IP address to check whether its service (HTTP) is UP or Down. I have created the following routes:

  1. /hosts?ip=IP or Domain – used to add new IP or domain to be checked
  2. /status – provides status from our IP/Domain

The following code handles HTTP requests to `/hosts?ip=`. Most important part is  $c->minion->enqueue(ping_ip => [$ip_address]); which means that every time an IP or Domain Name is given under that address, Minion will enqueue a task ping_ip and its arguments $ip_address

#/host?ip=ip_address
get '/host' => sub {
    my $c = shift;
    my $ip_address = $c->param('ip');
    $c->minion->enqueue(ping_ip => [$ip_address]);
    $c->render(text=> "OK: $ip_address");
};

Now that we have a procedure that enqueues a task, we need to implement ping_ip function under task handler (app->minion->add_task). Every time we enqueue a task (minion->enqueue), a unique job id is created. We will see this jobs later.

The following code receives the IP address and by using Net::Ping module, we check if port 80 is UP or DOWN, and we save the result at  ‘/tmp/report.txt’. As long as I understood from Mojolicious documentation, this procedure represents the Worker concept.

app->minion->add_task(ping_ip => sub {
    my ($job, $ip_address) = @_;

    $job->app->log->debug("Testing $ip_address");

    # 2 second timeout
    my $p = Net::Ping->new("tcp",2);
    $p->port_number(80);
    my $ret = $p->ping($ip_address);

    my $msg;
    if ($ret == 1) { $msg = "UP"; }
    else { $msg = "DOWN"; }

    my $filename = '/tmp/report.txt';
    open(my $fh, '>>', $filename) or die "Could not open file '$filename' $!";
    print $fh "$ip_address:$msg\n";
    close $fh;
});

The following opens ‘/tmp/report.txt’ file, and render it as json

get '/status' => sub {
    my $c = shift;
    my $filename = '/tmp/report.txt';
    open(my $fh, '<', $filename) or die "Could not open file '$filename' $!";
    my @lines = <$fh>;
    foreach my $entry (@lines) {
        chomp($entry);
        my ($key, $value) = split(/:/, $entry);
        $hosts_status{$key} = $value;
    }
    close($fh);
    $c->render(json => \%hosts_status);
};

Complete code:

use Mojolicious::Lite;
use Net::Ping;

plugin Minion => {Pg => 'postgresql://minionjobs:123456@localhost/minionjobs'};

my %hosts_status = ();

app->minion->add_task(ping_ip => sub {
    my ($job, $ip_address) = @_;

    $job->app->log->debug("Testing $ip_address");

    # 2 second timeout
    my $p = Net::Ping->new("tcp",2);
    $p->port_number(80);
    my $ret = $p->ping($ip_address);

    my $msg;
    if ($ret == 1) { $msg = "UP"; }
    else { $msg = "DOWN"; }

    my $filename = '/tmp/report.txt';
    open(my $fh, '>>', $filename) or die "Could not open file '$filename' $!";
    print $fh "$ip_address:$msg\n";
    close $fh;
});

#/host?ip=ip_address
get '/host' => sub {
    my $c = shift;
    my $ip_address = $c->param('ip');
    $c->minion->enqueue(ping_ip => [$ip_address]);
    $c->render(text=> "OK: $ip_address");
};

get '/status' => sub {
    my $c = shift;
    my $filename = '/tmp/report.txt';
    open(my $fh, '<', $filename) or die "Could not open file '$filename' $!";
    my @lines = <$fh>;
    foreach my $entry (@lines) {
        chomp($entry);
        my ($key, $value) = split(/:/, $entry);
        $hosts_status{$key} = $value;
    }
    close($fh);
    $c->render(json => \%hosts_status);
};

app->start;

It is time to run our application. I will use 4 Terminals

Running Mojolicious Site [Terminal 1]

$ morbo pingme2.pl 
Server available at http://127.0.0.1:3000

Lets add new tasks and enqueue them! [ Terminal 2]

$ curl localhost:3000/host?ip=google.com
$ curl localhost:3000/host?ip=facebook.com

From [Terminal 1], you can see some debug after queuing both names

[Sun May 21 22:18:49 2017] [debug] GET "/host"
[Sun May 21 22:18:49 2017] [debug] Routing to a callback
[Sun May 21 22:18:49 2017] [debug] 200 OK (0.015935s, 62.755/s)
[Sun May 21 22:19:03 2017] [debug] GET "/host"
[Sun May 21 22:19:03 2017] [debug] Routing to a callback
[Sun May 21 22:19:03 2017] [debug] 200 OK (0.002986s, 334.896/s)

Managing Jobs! [Terminal 3]

Listing all jobs status (active, inactive, finished, failed)

$ perl pingme2.pl minion job

Tasks 48 and 49 are both we have added. Task 47 was done before I wrote this post :D

49  inactive  default  ping_ip
48  inactive  default  ping_ip
47  finished  default  ping_ip

Getting queue stats

$ perl pingme2.pl minion job -s
{
  "active_jobs" => 0,
  "active_workers" => 0,
  "delayed_jobs" => 0,
  "enqueued_jobs" => 49,
  "failed_jobs" => 4,
  "finished_jobs" => 38,
  "inactive_jobs" => 2,
  "inactive_workers" => 0
}

Obtaining information from a task id

$ perl pingme2.pl minion job 49
{
  "args" => [
    "facebook.com"
  ],
  "attempts" => 1,
  "children" => [],
  "created" => "2017-05-22T02:19:03.70518Z",
  "delayed" => "2017-05-22T02:19:03.70518Z",
  "finished" => undef,
  "id" => 49,
  "parents" => [],
  "priority" => 0,
  "queue" => "default",
  "result" => undef,
  "retried" => undef,
  "retries" => 0,
  "started" => undef,
  "state" => "inactive",
  "task" => "ping_ip",
  "worker" => undef
}

Removing a job from queue

$ perl pingme2.pl minion job --remove 49
$ perl pingme2.pl minion job
48  inactive  default  ping_ip
47  finished  default  ping_ip

It is time to execute our jobs! Running Worker. [Terminal 4]

$ perl pingme2.pl minion worker
 [Sun May 21 23:09:38 2017] [debug] Worker 13109 started
 [Sun May 21 23:09:38 2017] [debug] Checking worker registry and job queue
 [Sun May 21 23:09:38 2017] [debug] Performing job "48" with task "ping_ip" in process 13113
 [Sun May 21 23:09:38 2017] [debug] Testing google.com

Adding new domain to ping it! Take a look at [Terminal 2] and [Terminal 4]

$ curl localhost:3000/host?ip=yahoo.com
Sun May 21 23:36:08 2017] [debug] Performing job "51" with task "ping_ip" in process 19676
[Sun May 21 23:36:08 2017] [debug] Testing yahoo.com

When you have Workers running, any new task is automatically performed .

Good luck with Minion

Source:

http://mojolicious.org/perldoc/Minion

http://mojolicious.org/perldoc/Minion/Command/minion/worker

http://mojolicious.org/perldoc/Minion/Command/minion/job

http://blogs.perl.org/users/brian_medley/2014/11/using-minion-with-a-rest-api.html

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.