Raids [BACKEND ONLY] #306

sebast1219 · 2017-06-26T08:29:07Z

Raid infos are implemented, feel free to add this fuctionnality on the frontend (I'm just using a personnal discord bot)

Table fort_raids
Fields :

id = Column(Integer, primary_key=True)
fort_id = Column(Integer, ForeignKey('forts.id'))
raid_seed = Column(Integer)
raid_battle_ms = Column(Integer, index=True)
raid_spawn_ms = Column(Integer, index=True)
raid_end_ms = Column(Integer, index=True)
raid_level = Column(Integer)
complete = Column(TINY_TYPE)
pokemon_id = Column(TINY_TYPE)
cp = Column(TINY_TYPE)
move_1 = Column(SmallInteger)
move_2 = Column(SmallInteger)

pokemon_id,cp,move_1 and move_2 are updated when raid battle starts

0.4 has not been released yet so use the latest commit from the develop branch instead.

Use asynchronous spawn cache in pogeo.monotools.

Remove dependence on shapely and geopy, improve performance of landmarks, automate landmark pickling/unpickling, read landmarks from config as a sequence of dicts instead of a Landmarks object, a little refactoring, miscellaneous fixes.

Add an example script for converting old-style landmarks configurations (constructing an actual Landmarks instance in config.py) with the new-style of using standard Python data-types (a tuple/list of dicts).

Use correct cache in add_mystery, use visit_point instead of visit in Overseer, fix setting spawn/despawn times in update_spawns, fix some variable names in Worker.

evenly-epic-mule · 2017-06-26T09:21:31Z

monocle/db.py

+    pokemon_id = Column(TINY_TYPE)
+    cp = Column(TINY_TYPE)
+    move_1 = Column(SmallInteger)
+    move_2 = Column(SmallInteger)


do the moves matter at all? as everyone will get something else after catching it

moves are the same for everyone in the battle against the raid boss. Thus it is a strategy part knowing if he has certain moves

you sure? what i heard is not same when in battle and after caught

I won't use it but that could be usefull to counter him

@walaoaaa1234 the moveset is the same for the raid boss battle but afterwards we get a randomized moveset for the capturable version.

evenly-epic-mule · 2017-06-26T09:22:05Z

monocle/db.py

+    raid_level = Column(Integer)
+    complete = Column(TINY_TYPE)
+    pokemon_id = Column(TINY_TYPE)
+    cp = Column(TINY_TYPE)


are the CP not the same for the same pokemon?

This is the CP of the boss ;) Not everytime the same imo

evenly-epic-mule · 2017-06-26T09:24:31Z

monocle/worker.py

+                            fort_raid['pokemon_id'] = fort.raid_info.raid_pokemon.pokemon_id
+                            fort_raid['cp'] = fort.raid_info.raid_pokemon.cp
+                            fort_raid['move_1'] = fort.raid_info.raid_pokemon.move_1
+                            fort_raid['move_2'] = fort.raid_info.raid_pokemon.move_2


these lines (after if fort.HasField('raid_info'):) should be part of normalize_raid

It already does...

I mean the code should be moved to the normalize_raid method
only leaving {} here

if fort.HasField('raid_info') and fort not in RAID_CACHE: db_proc.add(self.normalize_raid(fort_raid))

evenly-epic-mule · 2017-06-26T09:25:31Z

scan.py

@@ -27,7 +27,7 @@
 from monocle.utils import get_address, dump_pickle
 from monocle.worker import Worker
 from monocle.overseer import Overseer
-from monocle.db import FORT_CACHE
+from monocle.db import FORT_CACHE,RAID_CACHE


put a space in between :)

evenly-epic-mule · 2017-06-26T11:58:43Z

monocle/worker.py

+                            fort_raid['pokemon_id'] = fort.raid_info.raid_pokemon.pokemon_id
+                            fort_raid['cp'] = fort.raid_info.raid_pokemon.cp
+                            fort_raid['move_1'] = fort.raid_info.raid_pokemon.move_1
+                            fort_raid['move_2'] = fort.raid_info.raid_pokemon.move_2


I mean the code should be moved to the normalize_raid method
only leaving {} here

if fort.HasField('raid_info') and fort not in RAID_CACHE: db_proc.add(self.normalize_raid(fort_raid))

evenly-epic-mule · 2017-06-26T11:59:03Z

monocle/worker.py

+                        fort_raid['pokemon_id'] = ""
+                        fort_raid['cp'] = ""
+                        fort_raid['move_1'] = ""
+                        fort_raid['move_2'] = ""


these should be set to 0 as they are being saved as integer

j16sdiz · 2017-06-26T17:54:04Z

Can you do a rebase with whitespace / newline fix?
It is very difficult to review as is.

sebast1219 · 2017-06-27T10:07:01Z

Everything should be stable now. Tell me if you still have some troubles

NiliusJulius · 2017-06-27T10:35:49Z

I'm not sure why it happens, but when I run locally on my windows pc, it works fine.
However when I run this on my ubuntu server it doesn't write to the mysql database. when I stop scan.py it has a huge backlog of items it still wants to push to the DB.
It works fine without this addition however.

So it looks like it isn't able to write the data to the fort_raids table (because that stays empty) and that is clogging upp the writing to the DB.

sebast1219 · 2017-06-27T10:39:39Z

Can you drop fort_raid table, rerun create_db and retest it plz? (Don't forget to pull the lastest version of the PR)

NiliusJulius · 2017-06-27T13:45:22Z

Removing everything and reinstalling it all seems to have worked. However, the update function seems to be clogging it all up now.
On a fresh database everything works fine. but as soon as it tries to update an entry to include the raid boss, it blocks up the DB again.

NiliusJulius · 2017-06-27T14:32:13Z

monocle/db.py

+			cp = '{cp}',
+			move_1 = '{move_1}',
+			move_2 = '{move_2}',
+			notifDiscord = NULL


notifDiscord has to be removed in order for the update function to work.

sebast1219 · 2017-06-27T15:22:07Z

Corrected notifDiscord ;)
Sorry for that

j16sdiz · 2017-06-27T12:43:10Z

monocle/db.py

+    raid_spawn_ms = Column(Integer, index=True)
+    raid_end_ms = Column(Integer, index=True)
+    raid_level = Column(Integer)
+    complete = Column(TINY_TYPE)


This need to be boolean for postgresql

j16sdiz · 2017-06-27T12:43:37Z

monocle/db.py

+    __tablename__ = 'fort_raids'
+    id = Column(Integer, primary_key=True)
+    fort_id = Column(Integer, ForeignKey('forts.id'))
+    raid_seed = Column(String(50))


postgresql complain this

psycopg2.ProgrammingError: operator does not exist: character varying = bigint LINE 3: WHERE fort_raids.raid_seed = -6036655490970226155 AND fort_...

j16sdiz · 2017-06-27T12:44:37Z

monocle/db.py

+			cp = '{cp}',
+			move_1 = '{move_1}',
+			move_2 = '{move_2}',
+			notifDiscord = NULL


notifdiscord not exist in the table defination

baskettcase · 2017-06-27T18:56:37Z

I just want to make sure that this will work with SPAWN_ID_INT = false ?

baskettcase · 2017-06-27T19:21:21Z

Ok Im testing this out and Im getting a bunch of MySQL errors when the raid hasn't started yet since move_1, move_2, etc can not be "None" since they are smallint according to your specs, so the only thing that can be saved at this point are actual raid battles, not raid eggs.

Update 1: Ok I set it so that those fields could be NULL and that fixed that problem. My raid seed is a negative number.. is that correct?

sebast1219 · 2017-06-28T07:32:53Z

Yep the raid_seed is a negative number for what i saw

baskettcase · 2017-06-29T16:44:18Z

@sebast1219 Im unsure if I have the proper MySQL column types, do you have a create table that I can compare to? Also as I've been running this it doesn't seem like the raids are updated. I get when the initial raid is going to start, so all the moves, etc are NULL, but they never update to the actual raid info when it starts. I have a bunch of NULL in the database and only a very few that actually have the raid info after it has started, but I have a feeling that it was the first time it put those raids in the database and the raid was going when that happened.. so basically it doesn't seem to be updating the raid info. Also is there a way to keep track of when a raid is going to start and then deploy an account to get the raid info, just like a spawnpoint TTL?

baskettcase · 2017-06-30T18:46:21Z

@sebast1219 I know I asked on the gym one, but maybe a combo of gym and raid webhook? So when a gym changes teams and also when a raid is announced and starts?

Noctem and others added 30 commits April 28, 2017 01:01

WIP

7792ae7

WIP2

5321881

WIP3

b7d0ad1

Use pogeo's AltitudeCache

d2441b7

Update solve_captchas

ec1cc6f

DB improvements

9861703

Small Travis improvement

da68e70

WIP4

013d745

Update JS

b0cd60f

Update jquery to 3.2.1

b06b4fc

lint and clang-format main.js

88e5f2d

Update displaying scan area for new shape types

e8d4f31

Speed up PokéStop responses

90ad98b

Use AioSightingCache in web_sanic

7784e98

Update web.py for pogeo changes

6ca81f8

Skip tasks that are now done server-side

9b72cb7

Add favicon and tidy up HTML

9075028

Merge branch 'develop' into WIP

5ed1dcf

Change requirements to use pogeo develop branch

8ff947c

0.4 has not been released yet so use the latest commit from the develop branch instead.

Merge branch 'develop' into WIP

98b9cab

Merge branch 'develop' into WIP

90441a1

Use AioSpawnCache

d291663

Use asynchronous spawn cache in pogeo.monotools.

Use pogeo shapes instead of shapely for landmarks

e4baa9c

Remove dependence on shapely and geopy, improve performance of landmarks, automate landmark pickling/unpickling, read landmarks from config as a sequence of dicts instead of a Landmarks object, a little refactoring, miscellaneous fixes.

Add a script for converting landmarks config

b810e2b

Add an example script for converting old-style landmarks configurations (constructing an actual Landmarks instance in config.py) with the new-style of using standard Python data-types (a tuple/list of dicts).

Bug fixes

12d1a85

Use correct cache in add_mystery, use visit_point instead of visit in Overseer, fix setting spawn/despawn times in update_spawns, fix some variable names in Worker.

Merge branch 'develop' into WIP

0ea754e

Merge remote-tracking branch 'upstream/develop' into develop

aa6770d

c

b2d9ecd

Merge master

ff23ca3

6304

f687ebc

sebast1219 added 2 commits June 26, 2017 10:21

Reset branch to raid only

599e32e

Clean branch

441a9bc

evenly-epic-mule suggested changes Jun 26, 2017

View reviewed changes

sebast1219 added 2 commits June 26, 2017 16:41

Default values for raid to None

74c8967

Correcting cp type

0ce0822

sebast1219 added 2 commits June 27, 2017 10:37

Adding seed and correcting cache

c843568

Correcting update when pokemon spawns in raid

1aeba9a

raid_seed as varchar

3d43101

sebast1219 force-pushed the raids branch from 13d0f2d to 3d43101 Compare June 27, 2017 12:10

NiliusJulius reviewed Jun 27, 2017

View reviewed changes

Removed notifDiscord on update

77abfa5

j16sdiz suggested changes Jun 27, 2017

View reviewed changes

Sébastien Dixneuf added 5 commits June 29, 2017 11:39

Changed complete to boolean

1df901a

Changing complete type

f166a67

Testing boolean

ead87e8

Correcting raid_seed update

5aaaba1

Str of raid_seed

b5f4d8e

baskettcase mentioned this pull request Sep 11, 2017

Split out forks into their own classes Glennmen/PMSF#28

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Raids [BACKEND ONLY] #306

Raids [BACKEND ONLY] #306

sebast1219 commented Jun 26, 2017 •

edited

Loading

evenly-epic-mule Jun 26, 2017

thordurk91 Jun 26, 2017 •

edited

Loading

walaoaaa1234 Jun 26, 2017

sebast1219 Jun 26, 2017

thordurk91 Jun 26, 2017

evenly-epic-mule Jun 26, 2017

sebast1219 Jun 26, 2017

evenly-epic-mule Jun 26, 2017

sebast1219 Jun 26, 2017

evenly-epic-mule Jun 26, 2017

evenly-epic-mule Jun 26, 2017

evenly-epic-mule Jun 26, 2017

evenly-epic-mule Jun 26, 2017

j16sdiz commented Jun 26, 2017

sebast1219 commented Jun 27, 2017

NiliusJulius commented Jun 27, 2017

sebast1219 commented Jun 27, 2017

NiliusJulius commented Jun 27, 2017

NiliusJulius Jun 27, 2017

sebast1219 commented Jun 27, 2017

j16sdiz Jun 27, 2017

j16sdiz Jun 27, 2017

j16sdiz Jun 27, 2017

baskettcase commented Jun 27, 2017

baskettcase commented Jun 27, 2017 •

edited

Loading

sebast1219 commented Jun 28, 2017

baskettcase commented Jun 29, 2017 •

edited

Loading

baskettcase commented Jun 30, 2017

Raids [BACKEND ONLY] #306

Are you sure you want to change the base?

Raids [BACKEND ONLY] #306

Conversation

sebast1219 commented Jun 26, 2017 • edited Loading

Choose a reason for hiding this comment

thordurk91 Jun 26, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

j16sdiz commented Jun 26, 2017

sebast1219 commented Jun 27, 2017

NiliusJulius commented Jun 27, 2017

sebast1219 commented Jun 27, 2017

NiliusJulius commented Jun 27, 2017

Choose a reason for hiding this comment

sebast1219 commented Jun 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

baskettcase commented Jun 27, 2017

baskettcase commented Jun 27, 2017 • edited Loading

sebast1219 commented Jun 28, 2017

baskettcase commented Jun 29, 2017 • edited Loading

baskettcase commented Jun 30, 2017

sebast1219 commented Jun 26, 2017 •

edited

Loading

thordurk91 Jun 26, 2017 •

edited

Loading

baskettcase commented Jun 27, 2017 •

edited

Loading

baskettcase commented Jun 29, 2017 •

edited

Loading