Need help with Mojo::Dom and collections

[llarochelle]

Hi everyone, I'm trying to parse a webpage using Mojo::Dom and a ua. My ultimate goal is to mail part of it's content, but so far I'm wondering how I can use it's functions to populate a hashref for instance (not show in the code below). I managed to print out text using print or say, but if I try to map it to an array in a hashref, using ->each or ->join("\n") or even map->(text) it simply fails. When printing with Data::Dumper it shows a Mojo::Collection object and I can't seem to understand how to access it.

• How can I extract the text without the tags and use it as string ?

Here's some code

`
#!/usr/bin/perl

use local::lib '/home/luc/perl5/lib/perl5';
use Mojo::UserAgent;
use Mojo::DOM;
use utf8;
use v5.28;

my $url = 'https://www.aa.org/fr/daily-reflections';
my $ua = Mojo::UserAgent->new;
my $dom = $ua->get($url => {Accept => 'text/plain'})->res->dom;

my $main = $dom->at('div.reflection');
my $title = $main->find('h3 span.field--name-title')->join("\n");
my $strong .= $main->find('p strong')->join("\n");
for ($main->find('p strong')->each) {

    #       say $_;

}

print $title, $strong;

`

Thanks in advance for your help

[Tekki]

In the best case you can address a tag that surrounds the text you want to read and extract it in one step. As with

my $dom = $ua->get('https://mojolicious.org')->res->dom;

say $dom->at('.mojo-fortune')->all_text;

[llarochelle]

Hi Tekki, that's great thanks ! It seems like "find" and "at" can't be used the same way. So if I want to loop on elements and have some control, should I use a for loop or the each method ?

[Tekki]

$dom->at($selector) is equal to $dom->find($selector)->first, so the difference is not very big. What you overlooked is that these methods return Mojo::DOM objects or a collection of such object, not strings. You want to concatenate the texts inside these tags and 'have some control'. An example for this would be to get from mojolicious.org all the texts in <p> tags that contain the word 'Perl'. We could achieve this with

my $dom = $ua->get('https://mojolicious.org')->res->dom;
say $dom->find('p')->reduce(sub { $b->all_text =~ /Perl/ ? "$a\n" . $b->all_text : $a }, '');

[llarochelle]

Many thanks I will look into this and read the docs once more.

Cheers !