Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error otx-misp import with the latest MISP #37

Open
V1D1AN opened this issue Feb 15, 2020 · 39 comments
Open

Error otx-misp import with the latest MISP #37

V1D1AN opened this issue Feb 15, 2020 · 39 comments

Comments

@V1D1AN
Copy link

V1D1AN commented Feb 15, 2020

Description

Versions

  • otx-misp: 1.4.3
  • PyMISP: 2.4.121.1
  • MISP: 2.4.121
  • Python (Python 2 will not be supported): Python3
  • Operating system: Ubuntu 18.04.4 LTS

Traceback or error

When i execute:
otx-misp -o xxxxxxxxxxxxxxxxxx -s https://127.0.0.1 -m xxxxxxxxxxxxxxxxxxxxxxxx

I have always the error :

Error: 'TypeError' object has no attribute 'message'

@J-Robinson042519
Copy link

J-Robinson042519 commented Mar 3, 2020

Dang sorry to see this. I have the same issue. I also found -d works and with -vv it shows the OTX side is working but not the commit and -vv does not show anything more when -d is removed.

Name: pymisp
Version: 2.4.122

@JRobinsonNS
Copy link

JRobinsonNS commented Mar 3, 2020

switched to using cabby to pull from OTX and then use stix-to-misp.py to upload.

https://github.com/MISP/MISP-STIX-Converter

@V1D1AN
Copy link
Author

V1D1AN commented Mar 4, 2020

Hi,

I'm going to test this method. I don't know cabby, so it's going to be time to test the tool.

Can Cabby export in stix format?

Thanks for your idea

@J-Robinson042519
Copy link

It does stix by default. Here is what we are doing for this part.

We are also looking at contributing back to misp-stix-converter an option to add tags to the uploaded items.

@V1D1AN
Copy link
Author

V1D1AN commented Mar 4, 2020

Hi,

I have always the same error:
taxii-poll --path https://otx.alienvault.com/taxii/poll -c user_AlienVault --username MY_API_KEY --dest-dir /tmp

2020-03-04 20:25:03,098 INFO: Polling using data binding: ALL
2020-03-04 20:25:03,100 INFO: Sending Poll_Request to https://otx.alienvault.com/taxii/poll
2020-03-04 20:25:06,899 ERROR: FAILURE: There was a failure while executing the message handler

If I do this command:
taxii-poll -v --host otx.alienvault.com --https -c user_AlienVault --discovery /taxii/poll --username MY_API_KEY --dest-dir /tmp

Nota: I have add the verbose option. I have this logs:

2020-03-04 20:48:51,191 cabby.cli.poll INFO: Polling using data binding: ALL
2020-03-04 20:48:51,192 cabby.dispatcher INFO: Sending Discovery_Request to https://otx.alienvault.com/taxii/poll
2020-03-04 20:48:51,193 cabby.dispatcher DEBUG: Request:
<taxii_11:Discovery_Request xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="47964222-a925-40e6-98ca-35e360af7c7b"/>

2020-03-04 20:48:51,195 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): otx.alienvault.com:443
2020-03-04 20:48:54,443 urllib3.connectionpool DEBUG: https://otx.alienvault.com:443 "POST /taxii/poll HTTP/1.1" 200 441
2020-03-04 20:48:54,444 cabby.dispatcher DEBUG: Response:
<taxii_11:Discovery_Response xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="2749220666786904619" in_response_to="47964222-a925-40e6-98ca-35e360af7c7b">
<taxii_11:Service_Instance service_type="POLL" service_version="urn:taxii.mitre.org:services:1.1" available="true">
<taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
<taxii_11:Address>https://otx.alienvault.com/taxii/poll</taxii_11:Address>
<taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
<taxii_11:Message>OTX Taxii Polling</taxii_11:Message>
</taxii_11:Service_Instance>
<taxii_11:Service_Instance service_type="COLLECTION_MANAGEMENT" service_version="urn:taxii.mitre.org:services:1.1" available="true">
<taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
<taxii_11:Address>https://otx.alienvault.com/taxii/collections</taxii_11:Address>
<taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
<taxii_11:Message>OTX Taxii Collections</taxii_11:Message>
</taxii_11:Service_Instance>
<taxii_11:Service_Instance service_type="DISCOVERY" service_version="urn:taxii.mitre.org:services:1.1" available="true">
<taxii_11:Protocol_Binding>urn:taxii.mitre.org:protocol:https:1.0</taxii_11:Protocol_Binding>
<taxii_11:Address>https://otx.alienvault.com/taxii/discovery</taxii_11:Address>
<taxii_11:Message_Binding>urn:taxii.mitre.org:message:xml:1.1</taxii_11:Message_Binding>
<taxii_11:Message>OTX Taxii Discovery</taxii_11:Message>
</taxii_11:Service_Instance>
</taxii_11:Discovery_Response>

2020-03-04 20:48:54,449 cabby.client11.Client11 INFO: 3 services discovered
2020-03-04 20:48:54,450 cabby.dispatcher INFO: Sending Poll_Request to https://otx.alienvault.com/taxii/poll
2020-03-04 20:48:54,450 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Request xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="d61cf201-b82d-4ccd-80e9-61d02eff5fcb" collection_name="user_AlienVault">
<taxii_11:Poll_Parameters allow_asynch="false">
<taxii_11:Response_Type>FULL</taxii_11:Response_Type>
</taxii_11:Poll_Parameters>
</taxii_11:Poll_Request>

2020-03-04 20:48:54,452 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): otx.alienvault.com:443
2020-03-04 20:48:57,457 urllib3.connectionpool DEBUG: https://otx.alienvault.com:443 "POST /taxii/poll HTTP/1.1" 200 269
2020-03-04 20:48:57,458 cabby.dispatcher DEBUG: Response:
<taxii_11:Status_Message xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="6815088775670840839" in_response_to="d61cf201-b82d-4ccd-80e9-61d02eff5fcb" status_type="FAILURE">
<taxii_11:Message>There was a failure while executing the message handler</taxii_11:Message>
</taxii_11:Status_Message>

2020-03-04 20:48:57,459 cabby.cli.commons ERROR: FAILURE: There was a failure while executing the message handler
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/cabby/cli/commons.py", line 171, in run_client
run_func(client, args.uri, args)
File "/usr/local/lib/python3.6/dist-packages/cabby/cli/poll.py", line 139, in _runner
for counter, block in enumerate(blocks, 1):
File "/usr/local/lib/python3.6/dist-packages/cabby/client11.py", line 485, in poll
service_type=const.SVC_POLL)
File "/usr/local/lib/python3.6/dist-packages/cabby/abstract.py", line 190, in _execute_request
timeout=self.timeout)
File "/usr/local/lib/python3.6/dist-packages/cabby/dispatcher.py", line 63, in send_taxii_request
raise UnsuccessfulStatusError(obj)
cabby.exceptions.UnsuccessfulStatusError: FAILURE: There was a failure while executing the message handler

If you have any idea ....

@J-Robinson042519
Copy link

When using "taxii-poll --path https://otx.alienvault.com/taxii/poll -c user_AlienVault --username MY_API_KEY --dest-dir /tmp" the user_AlienVault does not have any alerts in it. Did you subscribe to someone and try to poll their feed? Also, the --host and --path were frustrating for me as well.

@V1D1AN
Copy link
Author

V1D1AN commented Mar 4, 2020

Yes, I have subscribe to the AlienVault user
The official of Alienvault.

I have change "taxii-poll --path https://otx.alienvault.com/taxii/poll -c AlienVault --username MY_API_KEY --dest-dir /tmp"

But i have the same error

@J-Robinson042519
Copy link

something is weird with that one. I just subscribed to user jnazario and I get the same error but go back to Metadefender and get a 504 then ran it again and got a 502 then again and it worked (bottom). It might be a busy server or gremlins in the interwebs or something.

taxii-collections --path https://otx.alienvault.com/taxii/collections --username MY_KEY --password foo
2020-03-04 21:19:34,189 INFO: Sending Collection_Information_Request to https://otx.alienvault.com/taxii/collections
=== Data Collection Information ===
Collection Name: user_AlienVault
Collection Type: DATA_FEED
Available: True
Collection Description: Data feed for user: AlienVault
Supported Content: All
=== Polling Service Instance ===
Poll Protocol: urn:taxii.mitre.org:protocol:https:1.0
Poll Address: https://otx.alienvault.com/taxii/poll
Message Binding: urn:taxii.mitre.org:message:xml:1.1

=== Data Collection Information ===
Collection Name: user_Metadefender
Collection Type: DATA_FEED
Available: True
Collection Description: Data feed for user: Metadefender
Supported Content: All
=== Polling Service Instance ===
Poll Protocol: urn:taxii.mitre.org:protocol:https:1.0
Poll Address: https://otx.alienvault.com/taxii/poll
Message Binding: urn:taxii.mitre.org:message:xml:1.1

=== Data Collection Information ===
Collection Name: user_jnazario
Collection Type: DATA_FEED
Available: True
Collection Description: Data feed for user: jnazario
Supported Content: All
=== Polling Service Instance ===
Poll Protocol: urn:taxii.mitre.org:protocol:https:1.0
Poll Address: https://otx.alienvault.com/taxii/poll
Message Binding: urn:taxii.mitre.org:message:xml:1.1

=== Data Collection Information ===
Collection Name: user_
Collection Type: DATA_FEED
Available: True
Collection Description: Data feed for user:
Supported Content: All
=== Polling Service Instance ===
Poll Protocol: urn:taxii.mitre.org:protocol:https:1.0
Poll Address: https://otx.alienvault.com/taxii/poll
Message Binding: urn:taxii.mitre.org:message:xml:1.1

taxii-poll --path https://otx.alienvault.com/taxii/poll --collection user_Metadefender --username <MY_KEY> --password foo -l 5 --dest-dir /tmp/otx-poll/
2020-03-04 21:29:01,803 INFO: Polling using data binding: ALL
2020-03-04 21:29:01,804 INFO: Sending Poll_Request to https://otx.alienvault.com/taxii/poll
2020-03-04 21:29:37,822 INFO: Content block saved to /tmp/otx-poll/user_Metadefender_d3c4a5e052b7a89cfbf5b627207ced11
2020-03-04 21:29:37,999 INFO: Content block saved to /tmp/otx-poll/user_Metadefender_dae6ff33b6bcb2921129a307bb4a8333
2020-03-04 21:29:38,279 INFO: Content block saved to /tmp/otx-poll/user_Metadefender_b08ee881668b25a99a864f51c88fbd1f
2020-03-04 21:29:38,564 INFO: Content block saved to /tmp/otx-poll/user_Metadefender_9a4817a0f425daa605943a3cd6af1ba9
2020-03-04 21:29:38,725 INFO: Content block saved to /tmp/otx-poll/user_Metadefender_13bc24ef63e945c99ddb15339c89b23f
2020-03-04 21:29:38,725 INFO: 5 blocks polled

@V1D1AN
Copy link
Author

V1D1AN commented Mar 4, 2020

I have subscribe to the user popularmalware and it's work for me...
But if I test with the user_Alienvault .. I have the error .. grrrrr

perhaps a problem with the Alienvault user ..

@J-Robinson042519
Copy link

Good to hear it worked for you. Maybe we need to take the debug to Alienvault.

@rustybrooks
Copy link

Hi,

I work on the OTX team. I'll look into this as soon as I can and get back to you.

@V1D1AN
Copy link
Author

V1D1AN commented Mar 5, 2020

hi,
can you see for otx-misp and why we can’t poll with the collection user_AlienVault .. I know that that last problem doesn't belong in this github.

@rustybrooks
Copy link

I don't know too much about otx-misp but it's probably the same problem that occurs with Cabby, which is that there's an error encountered when trying to render the AlienVault user's feed to STIX.

@rustybrooks
Copy link

rustybrooks commented Mar 6, 2020

OK, so, I believe that I have fixed the problem related to not being able to get TAXII results for the collection "user_AlienVault" (and possibly some other users had the sample problem, the error was related to converting some types of data to STIX and only affected users who had examples of that kind of data)

If that doesn't fix the otx-misp problems let me know and I can look deeper into it.

As an aside - if you're using Cabby for anything other than testing I would recommend limiting the results by date. The AlienVault user has a LOT of pulses/indicators, and not providing a date means that it downloads all of them, which is quite slow/taxing. Providing a date will only get pulses that are new or have been modified since that date, so if you know you fetched N hours ago, then limiting to that time frame will greatly reduce fetch time and traffic.

Such as
$ taxii-poll --path https://otx.alienvault.com/taxii/poll -c user_AlienVault --username YOURKEYHERE --password foo --dest-dir /tmp --begin 2020-03-01

Ideally you'd use --begin with whatever your last fetch date was, and just keeping feeding "new" indicators into whatever you're using.

@V1D1AN
Copy link
Author

V1D1AN commented Mar 6, 2020

yes it works :)
I will keep your username if i have another problem with OTX :)
Where we push this discussion for other persons ??

@V1D1AN
Copy link
Author

V1D1AN commented Mar 6, 2020

Thanks J-Robinson042519 for misp-stix-converter

I have changed "/usr/local/lib/python3.6/dist-packages/misp_stix_converter/converters/convert.py"

I added:
misp_event.add_tag('OTX')
misp_event.add_tag('tlp:white')
misp_event.publish()

Thanks a lot J-Robinson042519 and rustybrooks

@J-Robinson042519
Copy link

@V1D1AN - did you commit this to the converter project? If not please do or make it a config parameter and add it to the project. I don't think we will get to it for a while.

Does anyone know how to limit the polling on otx-misp like @rustybrooks did for taxii-poll?

@JRobinsonNS
Copy link

looks like with the fix from @rustybrooks did not help otx-misp...still getting Error: 'TypeError' object has no attribute 'message'. One theory we have is it is coming from MISP with its Py 2.7 conversation issues.

@J-Robinson042519
Copy link

Thanks J-Robinson042519 for misp-stix-converter

I have changed "/usr/local/lib/python3.6/dist-packages/misp_stix_converter/converters/convert.py"

I added:
misp_event.add_tag('OTX')
misp_event.add_tag('tlp:white')
misp_event.publish()

Thanks a lot J-Robinson042519 and rustybrooks

@V1D1AN - Where in the code did you add the misp_event section? around like 210 looks about right.

@davecabio
Copy link

davecabio commented Mar 27, 2020

Good morning,
I'm having the same problem. How did you solved it?
Will the bug be resolved in otx_misp?

MISP version: 2.4
Command:
otx-misp --otx "MYKEY" --server "https://localhost" --misp "MYKEY" --discover-tags -n -a --threat-level high -t "2020-03-26T19:20:30+01:00" -vv
Result:
Error: 'TypeError' object has no attribute 'message'

If I add -d, it works perfectly.

@davecabio
Copy link

OK, so, I believe that I have fixed the problem related to not being able to get TAXII results for the collection "user_AlienVault" (and possibly some other users had the sample problem, the error was related to converting some types of data to STIX and only affected users who had examples of that kind of data)

If that doesn't fix the otx-misp problems let me know and I can look deeper into it.

As an aside - if you're using Cabby for anything other than testing I would recommend limiting the results by date. The AlienVault user has a LOT of pulses/indicators, and not providing a date means that it downloads all of them, which is quite slow/taxing. Providing a date will only get pulses that are new or have been modified since that date, so if you know you fetched N hours ago, then limiting to that time frame will greatly reduce fetch time and traffic.

Such as
$ taxii-poll --path https://otx.alienvault.com/taxii/poll -c user_AlienVault --username YOURKEYHERE --password foo --dest-dir /tmp --begin 2020-03-01

Ideally you'd use --begin with whatever your last fetch date was, and just keeping feeding "new" indicators into whatever you're using.

May you help us? As I'm seeing I'm not the only one with this issue.

Thanks!

@rustybrooks
Copy link

I've actually been playing with otx-misp today, and I have the same problem as you. I think what it comes down to is that otx-misp relies on another library called pymisp. The most recent version of pymisp has a PyMISP() object that does not take the same init parameters as it used to, so otx-misp fails.

I've tried downgrading to an older version of pymisp, and that gets past this problem, but there are other problems down the road. pymisp is handling the MISP api calls and my guess is that how these calls works has changed.

If I update to a newer pymisp and fix the initialization problems then I run into the fact that pymisp uses different functions - the "new_event" function no longer exists for example. So I think it's fixable but possibly not trivial to do so. I'm still looking into it, and other alternatives such as offering a MISP feed directly from OTX

@rustybrooks
Copy link

rustybrooks commented Mar 30, 2020

Note, a fork of otx-misp has fixed this problem I believe:
https://github.com/obert01/otx_misp

I myself did not go down this road. Here's the problem as I see it

  1. MISP and pymisp (which otx-misp uses) are tightly coupled - you generally need a newish version of pymisp to interact with MISP or things don't work
  2. pymisp underwent a fairly large change to how it works a few months ago - it created enough incompatibility that it's a pretty big job to update otx-misp - look at the last commit in the repo above. It's a bit rude to have an API change this large in such a minor release of pymisp
  3. because MISP and pymisp are tightly coupled you can't even just roll back to an older version of pymisp, because it won't work with new MISP server installs.
  4. because of this, even though it's now fixed in the repo above, that repo might not work with older MISP servers.

I kind of gave up on fixing this and I'm going to instead, I think, make MISP feeds for OTX. I did the first one here, which lets you get a MISP feed from a group, as a POC
https://otx.alienvault.com/api/v1/groups/GROUPID/misp_feed
you can add this as a feed in MISP, you'll need to give
X-OTX-API-KEY: YOURKEYHERE
in the headers box.

I will probably add more MISP feed support in the near future

Note: edited to fix URL

@Tubbs10
Copy link

Tubbs10 commented Apr 5, 2020

Note, a fork of otx-misp has fixed this problem I believe:
https://github.com/obert01/otx_misp

I myself did not go down this road. Here's the problem as I see it

  1. MISP and pymisp (which otx-misp uses) are tightly coupled - you generally need a newish version of pymisp to interact with MISP or things don't work
  2. pymisp underwent a fairly large change to how it works a few months ago - it created enough incompatibility that it's a pretty big job to update otx-misp - look at the last commit in the repo above. It's a bit rude to have an API change this large in such a minor release of pymisp
  3. because MISP and pymisp are tightly coupled you can't even just roll back to an older version of pymisp, because it won't work with new MISP server installs.
  4. because of this, even though it's now fixed in the repo above, that repo might not work with older MISP servers.

I kind of gave up on fixing this and I'm going to instead, I think, make MISP feeds for OTX. I did the first one here, which lets you get a MISP feed from a group, as a POC
https://otx.alienvault.com/api/groups/GROUPID/misp_feed
you can add this as a feed in MISP, you'll need to give
X-OTX-API-KEY: YOURKEYHERE
in the headers box.

I will probably add more MISP feed support in the near future

Hi,

How am I supposed to get this feed working? Is the URL really GROUPID? How about source format in MISP? Is it MISP Feed then? For what I tested it didn't work with this information.

Thank you.

@rustybrooks
Copy link

GROUPID should be the numeric group id of the group you want to sync. For example this is a newly created public group:
https://otx.alienvault.com/group/867/pulses

It's a MISP feed, i.e. it has a /manifest.json and then a json file per pulse in the feed.

This url would be to sync a single group. We'll be adding the ability to sync a user and probably also your subscription

@AntoninHL
Copy link

Hi,
thanks @rustybrooks for the alternative method, but I always have an Failed Job when I try to setup a MISP Feed with https://otx.alienvault.com/api/groups/GROUPID/misp_feed with X-OTX-API-KEY: YOURKEYHERE in the headers box.

On OTX Side, I created a group with 2 pulses.

Any idea?

Thanks a lot !

@rustybrooks
Copy link

In that URL you need to replace GROUPID with the integer group id for your group. For example if you go to a group in the UI, the url will look something like
https://otx.alienvault.com/group/927/pulses
and the GROUPID value would be replaced with 927 to be like
https://otx.alienvault.com/api/groups/927/misp_feed

And of course YOURKEYHERE in the headers would have to be your OTX api key.

If you have access to the MISP logs I think you'll find errors in there. I'm not currently running a MISP instance but I could maybe fire one up again if necessary to try to check it out.

@AntoninHL
Copy link

Thanks @rustybrooks .

I had, of course, replaced all the values to be changed. ;-)

Here are logs:

2020-05-23 18:52:53 Error: Could not get new event uuids for feed 69. [Exception] Fetching the 'https://otx.alienvault.com/api/groups/928/misp_feed/manifest.json' failed with HTTP error 404: Not Found Stack Trace: #0 /var/www/MISP/app/Model/Feed.php(194): Feed->feedGetUri(Array, 'https://otx.ali...', Object(HttpSocket), true) #1 /var/www/MISP/app/Model/Feed.php(137): Feed->downloadManifest(Array, Object(HttpSocket)) #2 /var/www/MISP/app/Model/Feed.php(845): Feed->getNewEventUuids(Array, Object(HttpSocket)) #3 /var/www/MISP/app/Console/Command/ServerShell.php(213): Feed->downloadFromFeedInitiator('69', Array, '4542') #4 /var/www/MISP/app/Console/Command/AppShell.php(35): ServerShell->fetchFeed() #5 /var/www/MISP/app/Vendor/kamisama/php-resque-ex/lib/Resque/Job.php(199): AppShell->perform() #6 /var/www/MISP/app/Vendor/kamisama/php-resque-ex/lib/Resque/Worker.php(278): Resque_Job->perform() #7 /var/www/MISP/app/Vendor/kamisama/php-resque-ex/lib/Resque/Worker.php(241): Resque_Worker->perform(Object(Resque_Job)) #8 /var/www/MISP/app/Vendor/kamisama/php-resque-ex/bin/resque(109): Resque_Worker->work('5') #9 /var/www/MISP/app/Vendor/kamisama/php-resque-ex/bin/resque(100): startWorker('default', 1, Object(MonologInit\MonologInit), '5') #10 {main}

@rustybrooks
Copy link

Aw man sorry I had a typo in the URL, it should be
https://otx.alienvault.com/api/v1/groups/69/misp_feed
I'll see if I can edit my earlier post

@AntoninHL
Copy link

👌 Perfect!

It works like a charm!

Many thanks for this workaround @rustybrooks!

I hope we will have soon a new version of otx-misp.

@seldem
Copy link

seldem commented Jun 18, 2020

Thanks for this @rustybrooks . It is very nice to have OTX pulses as MISP events. But I guess I found something to report you: In MISP there are "email-dst" and "email-src" attributes. As I understood in OTX only one attribute exist which is "email". When I checked OTX events in my MISP they are always "email-dst". Can you fix it? I believe OTX "email" should map to MISP "email-src".
Thanks in advance.

@rustybrooks
Copy link

Yeah, I can change that, sure. It probably does make more sense for it to be email-src which I guess is probably the "from" address

@martinconnarty
Copy link

Thanks @rustybrooks for this well needed solution. I am quite new to using OTX, but have seen that many authors create new pulses every day, rather than updating an existing one. Unless I'm mistaken, this means that me adding that pulse to a Group is good for that day and no more. Have you managed to explore putting similar MISP manifests on either Users, or just the subscribed pulses? I have had no end of difficulties with trying to get OTX-MISP working which I acknowledge is probably mainly user error/ignorance, this solution would be invaluable to me!

@rustybrooks
Copy link

Yeah there are plans to add MISP support to users, and/or making a MISP feed for your "subscription" which includes:

  • all public pulses from users you're subscribed to
  • all pulses from groups
  • and then subtracting all pulses that you've specifically excluded

Might come through before the end of this year but is not immediately planned

@tonycaud
Copy link

The error 'TypeError' object has no attribute 'message' is happening again...it started very recently for me. In fact, I was receiving an error message stating that I had to move to the latest version of MISP to continue, then after I did, I started getting these TypeError messages.

I've been trying to find a way around this and ended up here.

AFAIK, we were pulling all public pulses in the primary Alienvault feed (I believe that means I'm pulling all public pulses from user Alienvault). There doesn't appear to be a group number for that main public feed. If there is, please let me know as I would love to try getting the feeds using this other method.

We are using a config file, but running it from the manually from the CLI with the verbose switch provided no additional information.

Any help would be appreciated.

@rustybrooks
Copy link

@tonycaud have you tried the fork from this project I mentioned above?

@tonycaud
Copy link

Since I simply installed using "pip3 install otx-misp" how do I see which I have?

@rustybrooks
Copy link

Then you have this one, almost surely. I think you'd have to download the fork from github and install it manually.

@tonycaud
Copy link

tonycaud commented Feb 23, 2021

So, that one throws a chunk of errors, and there isn't an issues page there...

-nevermind...with help from a friend of mine, we changed it from calling Python2.7 to Python3 and now it seems to be working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants