adding predatory conversations to PAN2012

[rezaBarzgar]

The PAN dataset is for 2012. We know that predatory conversations of PAN are from perverted-justice. However, Here are some conversations that had been added after 2012.
We should check these conversations. If they are not in the dataset, I think it's a good idea to add them.

[hosseinfani]

@rezaBarzgar
Nice catch. Do you think this would be a task for @EhsanSl , that is writing a crawler and creating a new dataset with the same XML format as the old one?

[rezaBarzgar]

@hosseinfani
I think that's a good idea. These conversations can be useful.

[rezaBarzgar]

@EhsanSl
Hi Ehsan, I assigned you to this issue.

The objective is to expand the dataset with conversations that are not in PAN2012. As Dr. Fani mentioned, you must write a crawler that extracts new data from perverted justice with the same XML format.

We can have a meeting if you need any information.

Please submit your progress on this issue

[EhsanSl]

nice try guys! 🥲

[EhsanSl]

Hi dear Reza, I hope everything is going well brother,
I haven't forgotten about the tasks, but frankly, I'm extremely swamped this week, and have a few too many deadlines, I'll make up for it from the reading week. x)
 

[EhsanSl]

Hi dear Dr. Hossein and dear Reza
I attached the crawler here neuralcrawing_.zip
, there are a few points that are worth mentioning
[]since in all the chat logs, the conversations are not separated by a distinct box, there was not a definite way to extract each individually (to my understanding),
[] the date is sometimes placed inside the conversation div and sometimes between them, which makes it a bit challenging
[] the time formatting is not consistent for all the convicts
[] to access the crawler file: neuralcrawling/neuralcrawling/spiders/justice_spider.py
[] to run the crawler, make sure the terminal path is: 'neuralcrawling/neuralcrawling'
then run the following command in the terminal: scrapy crawl jspider -o output.json