Merge branch 'main' of https://github.com/adir1/adir1hugo

content/posts/2024-algorithmic-trading-deeper-dive/index.md

@@ -0,0 +1,73 @@
+---
+title: Algorithmic Trading - Deeper technical dive for Dark Pool swimmers.
+description: Deeper dive into Quants, Algorithmic trading concepts and more. Exploring prediction models, financial crash, Black Swan book, Options and more. 
+author: Adi Rabinovich
+type: post
+heroStyle: "background"
+showTableOfContents: true
+draft: true
+date: 2024-07-11T11:11:11+00:00
+url: /2024/algorithmic-trading-deeper-dive/
+tags:
+  - Markets
+  - Economics
+  - Quants
+  - AI
+  - Derivatives
+  - Options
+
+---
+## TLDR
+
+This is deeper dive - if you need introduction to the topic, [check out first post in this series](/2024/swimming-in-the-dark-pools-quants-ai-trading/).
+
+## Technical Analysis - Gaps, Tea-Cups, The Whole Darn Party!
+
+## Artificial Intelligence Knows All (Digital Twins Syndrome)
+
+Monitoring endless inputs is expensive. And how good are our digital twins now? The concept popularized by nVidia but explored under many names much earlier, by our entire industry (research more on the origin of 'digital twin' expression)
+
+And a lot of it depends on our goals - is it Dividend, is it Growth, how much growth/dividend?
+
+
+## Ethics of *Market Liquidity*
+
+When sh*t hits the fan, all automated systems stop buying also... So where is the liquidity?
+
+## KISS Principle (Keep it Simple _Silly_)
+
+This topic is huge, technical charting is super popular and not fully automated as far as I know. AI analysis of many sources. More Advanced post is coming - to dive deeper into those issues for those interested.
+
+
+## Notes
+
+Derivatives in general - are they useful after collapse of 2007/2008? And what about Futures? or Commodities trading?
+Black Swan book mention
+
+https://www.bbc.com/news/explainers-51265169
+
+
+## OLD SAMPLE
+
+I don't claim to have all the answers, these are hard problems that will take time to solve. For starters it is important that we understand our limitations when making decisions based on data. Last thing Netflix wants is disgruntled consumers - some get so emotionally invested that they may swear-off Netflix from young age!
+
+1. Focus on continuously improving data collection, with real-world testing to confirm accuracy. All analysis is only as good as the input data it gets!
+2. Don't be afraid of creative approaches to confirm assumptions. Actually ask in the app - would you like to remove this show from Continue Watching because you didn't like it?
+3. Discovery is still a huge problem - find ways to show trailers even to non-Ads consumers. Offer bigger banners for new releases with some teasers. Be creative here - and ensure all analysis of popularity takes into account how "discoverable" and "promoted" this new content was in the first place! And why not have Netflix's own annual Awards show for original content - cheap enough to produce and viewers can quickly get a taste of "What's new and good" on the platform?
+4. People get invested into seeing stories to conclusion - this is both Useful for content producers and a possible trap. One solution for expensive productions is to make a single-story movie (perhaps with underlying hints for a bigger "conspiracy"), and if successful make additional movies in the same Universe/Series.
+5. Consider the cost from the get-go - I am sure productions get green-light based on some sort of statistical analysis for how many subscribers like similar content and thus potential viewers. But how accurate is this - if director isn't the same, or actors differ, it can have huge impact on seemingly similar script. And what about Potential subscribers - surely Netflix needs to entice people who look at the catalog today and say "nothing interesting for me here"...
+
+In a recent Interview for Bloomberg, Netflix new co-CEOs made a bold statement that "[they never cancel a successful show](https://www.forbes.com/sites/paultassi/2023/01/24/netflix-says-it-has-never-cancelled-a-successful-show/)". This statement quickly catapulted to the top of the news by itself, simply because most of Netflix subscribers vehemently disagree with it! In 
+![Netflix Viewers In Pain](people_crying_for_netflix_cancellations.png)
+
+## Some related links to explore
+
+- [Netflix Shows Cancelled in 2022](https://variety.com/lists/netflix-shows-canceled-2022/)
+- [Interview with Netflix new Co-CEOs - a window into their thinking](https://www.bloomberg.com/news/newsletters/2023-01-21/netflix-ceo-reed-hastings-steps-down-interview-with-greg-peters-ted-sarandos)
+- [Netflix push into video games](https://www.theverge.com/22772589/netflix-video-games-app-news-updates) - I plan to blog in the future on this Exciting effort
+- [How We Decide by Jonah Lehrer](https://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0547247990?tag=craftonia-20)
+- [Predictably Irrational](https://www.amazon.com/Predictably-Irrational-Revised-Expanded-Decisions/dp/0061353248?tag=craftonia-20) - Expanded edition of the classic
+
+{{< alert "image" >}}
+**Images By DALL-E 3 from Microsoft Bing**
+{{< /alert >}}

content/posts/2024-introducing-merka-cloud/index.md

@@ -16,15 +16,61 @@ tags:
   - IaC
   - AI
   - ML
-
+  - AWS
+  - GCP
+  - Azure
+  - VCS
+  - Database
+  - Business
+  - 
 
 ---
 
+## TLDR
+
 
 
+## Notes
 
+- Describe mindset shift - where does one start for new business, what about existing business? VCS is the root? Cloud provider? Systems? NO - People is the true Root, and hence IDENTITY is the true root. Mobile APP is the ROOT - Human starting point! Org(s) are by far secondary. ROLES are at the root, part of the platform, AI aware.
+- WHY: Most costs actually go into People rather than computing resources - with drastic reduction in license costs and hosting costs, unless you have major AI computing needs, you need to focus on people cost
+- Consultants and experts are expensive - all the free software needs integration, this is the "Catch"
+- WHY: Supply chain attacks will only Escalate in the future, Huge Security Issue but also opportunity! Security over privacy - AGAIN!
+- WHY: ZeroTrust starting point - degrees of ZeroTrust, overheads, identity again
+- WHY: Marketplace Templates as a starting point - Marketplace hosted on merka.cloud, everything else self-hosted - EASY TO USE - MARKET downloads tracked via ClientID somehow?
+- AI that understands Your Cloud
+- Hybrid AI solution that saves a lot of energy by: Analyzing complex verbal queries into relevant specific templates (with dynamic data from SQL/APIs in them) and then caching this AI analysis to avoid redoing it next time completely
+- Major part of the solution is simple CHAT platform (like Slack, Teams or Discord), with optional AI product that tracks all discussions and can answer questions about them, including summarization based on time period. Also AI reads and summarizes/learns any posted links or embedded documents. Can also integrate to external chat platforms for same functionality.
+- WHY: Self Hosted is the big goal - on Bare Metal or on VMs
+- Helps figure out pricing of bare-metal or VMs, helps track performance of Networks, learns physical topology vs SDN topology (ZeroTrust)
+- No Server Names at all - ONLY Services via DNS, everything and anything
+- Identities should be connected to Roles at AI level, we can ask it also including evolution
+- Evolution is huge built-in element, obviously. This is including people that join/leave, contribution tracking for self-funding startup, obviously versioning of code, infrastructure and data is at the core - declarative infrastructure and business
+- Rise from the Ashes principle - rebuilding any portions of infrastructure should be seamless and easy, whether to move providers or to overhaul/upgrade
+- Most of the data is encrypted at rest - but PII should be more encrypted/scrambled, what about actual keys and tokens? Security needs major design from the ground up - again ZeroTrust and PHONE is required
+- Perhaps App should be required part of the solution - higher role can reactivate but essentially it is major red flag that employee can't keep themselves safe and can't properly migrate to new device themselves. Should always full destroy all access if moving devices, also GPS is required part of solution - hence App requirement. Security above privacy - no compromises.
+- WHY: Surprise billing is huge issue, and even bigger issue is Moving from cloud to cloud - but it shouldn't be. It should even be possible to run a business on Multiple clouds at the same time, and then turn On/Off various clouds if unhappy with pricing or for any reason really.
+- Nobody can self-delete logs - history tracking is at the core of the project, including logs. Investigate Blockchain-like guarantees for integrity - nothing can be "disappeared", even if past history can be changed, much like in accounting - backdated operation arriving in the future.
+- WHY: NO LOCK IN - Pluggable/Service-Providers Architecture at the core. Even IDENTITY APP should have a shared Protocol. Everything is Protocol in Fact, and AI is trained to know all file types, API types, protocols and more. Technology Stack(share) doghouse, migration is hell. 
+- CDN Converging on serving files really well, including Web Assets, Images, Video. Arguably WebPack should not be needed - leave it also to CDN? Should CDN be at the core of the solution - or only via Pluggable-Service?
+- Offline caching of images, data, etc - self guiding (based on frequency of modification) and fluid cache/no-cache behavior. Sync to Offline built-in, generally sync is important - there should be screens and ways of seeing it and solving it.
+- Real-time auto-generating UI Forms or Sheets - including graphs, charts and more - AI generated when needed, since all data available
+- VERSIONING At the heart, including data
+- Data Lineage built-in, tracking transformations and visualization should also be possible
+- Code Build, Orchestration, all La-Natural - File system tracks what has been build, generally builds are automatic and configurable post certain timed delay.
 
-Surprise billing is huge issue, and even bigger issue is Moving from cloud to cloud - but it shouldn't be. It should even be possible to run a business on Multiple clouds at the same time, and then turn On/Off various clouds if unhappy with pricing or for any reason really.
 
 Surprise Billing and Moving Clouds discussion: https://news.ycombinator.com/item?id=41312834
 
+WATCH FOR NETWORKING: https://www.youtube.com/watch?v=SYeq1fDcUl0
+
+
+More and more integrations - where is marketplace? Security governance? What goes on?
+https://www.infracost.io/
+
+https://spacelift.io/integrations
+https://idx.dev/blog/article/a-year-of-project-idx
+https://coder.com
+
+Learn LVM, ext4, some other new -Nix FS options especially with built-in versioning (consider LakeFS, Git, etc)
+Stack Share Doghouse

content/posts/2024-public-cloud-ease-of-use/index.md

@@ -0,0 +1,63 @@
+---
+title: Public Cloud - A Huge Leap Forward (or is it?)
+description: Public Cloud Ease of Use has been completely neglected and continues to get worse. We had such tremendous evolution of hosting tools over the years, with cPanel and more. And now cloud dashboards are overly complex instead.
+author: Adi Rabinovich
+type: post
+heroStyle: "background"
+showTableOfContents: true
+draft: true
+date: 2024-09-01T11:11:11+00:00
+url: /2024/2024-public-cloud-ease-of-use/
+tags:
+  - Cloud
+  - Hosting
+  - Development
+  - Evolution
+  - AWS
+  - GCP
+  - Azure
+
+---
+## TLDR
+
+Somehow we went backwards over the recent years with endless complexity of public cloud dashboards. Obviously public clouds are in direct competition and don't want you to be able to easily switch providers, but this way everybody loses! GCP was the first to try radical new ideas with GAE, but sadly that backfired because devs like the comforting and familiar. Today most clouds are VMs and Containers, but where is the standard? Where is the ease of use? Where is the marketplace?
+
+## Introduction to Algorithmic Trading and Quants
+
+
+## Notes
+Focus on how we abandoned simplicity - we should have powerful features available, but hidden behind more advanced options. More wizards and screens for simple scenarios.
+Market deployment should be more powerful - provisioning wise and explaining costs
+Backup, scaling, security and networking are probably biggest complicating factors - especially that de-facto cloud assumes not to know anything about your existing resources and account
+Review and mention: Terraform, OpenTofu, Pulumi, AWS CloudFormation, Ansible, Crossplane, Firefly.IO, Spacelift, Fly.io
+Tease that something coming from my own ideas on this!
+
+https://www.cloudpanel.io/blog/hosting-panels/
+https://spacelift.io/blog/terraform-alternatives
+
+
+
+## OLD SAMPLE
+
+I don't claim to have all the answers, these are hard problems that will take time to solve. For starters it is important that we understand our limitations when making decisions based on data. Last thing Netflix wants is disgruntled consumers - some get so emotionally invested that they may swear-off Netflix from young age!
+
+1. Focus on continuously improving data collection, with real-world testing to confirm accuracy. All analysis is only as good as the input data it gets!
+2. Don't be afraid of creative approaches to confirm assumptions. Actually ask in the app - would you like to remove this show from Continue Watching because you didn't like it?
+3. Discovery is still a huge problem - find ways to show trailers even to non-Ads consumers. Offer bigger banners for new releases with some teasers. Be creative here - and ensure all analysis of popularity takes into account how "discoverable" and "promoted" this new content was in the first place! And why not have Netflix's own annual Awards show for original content - cheap enough to produce and viewers can quickly get a taste of "What's new and good" on the platform?
+4. People get invested into seeing stories to conclusion - this is both Useful for content producers and a possible trap. One solution for expensive productions is to make a single-story movie (perhaps with underlying hints for a bigger "conspiracy"), and if successful make additional movies in the same Universe/Series.
+5. Consider the cost from the get-go - I am sure productions get green-light based on some sort of statistical analysis for how many subscribers like similar content and thus potential viewers. But how accurate is this - if director isn't the same, or actors differ, it can have huge impact on seemingly similar script. And what about Potential subscribers - surely Netflix needs to entice people who look at the catalog today and say "nothing interesting for me here"...
+
+In a recent Interview for Bloomberg, Netflix new co-CEOs made a bold statement that "[they never cancel a successful show](https://www.forbes.com/sites/paultassi/2023/01/24/netflix-says-it-has-never-cancelled-a-successful-show/)". This statement quickly catapulted to the top of the news by itself, simply because most of Netflix subscribers vehemently disagree with it! In 
+![Netflix Viewers In Pain](people_crying_for_netflix_cancellations.png)
+
+## Some related links to explore
+
+- [Netflix Shows Cancelled in 2022](https://variety.com/lists/netflix-shows-canceled-2022/)
+- [Interview with Netflix new Co-CEOs - a window into their thinking](https://www.bloomberg.com/news/newsletters/2023-01-21/netflix-ceo-reed-hastings-steps-down-interview-with-greg-peters-ted-sarandos)
+- [Netflix push into video games](https://www.theverge.com/22772589/netflix-video-games-app-news-updates) - I plan to blog in the future on this Exciting effort
+- [How We Decide by Jonah Lehrer](https://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0547247990?tag=craftonia-20)
+- [Predictably Irrational](https://www.amazon.com/Predictably-Irrational-Revised-Expanded-Decisions/dp/0061353248?tag=craftonia-20) - Expanded edition of the classic
+
+{{< alert "image" >}}
+**Images By DALL-E 3 from Microsoft Bing**
+{{< /alert >}}

content/posts/2024-swimming-in-dark-pools/index.md

@@ -23,7 +23,7 @@ I discovered the amazing world of algorithmic trading probably about 10 years ag
 
 ## Introduction to Algorithmic Trading and Quants
 
-
+Recommend the Udemy class I like: [Python for Financial Analysis and Algorithmic Trading](https://www.udemy.com/share/101XQ23@mNHHN4xc2oyh4hhICKUa5YbsjY6WhCFkBZ4ZLVHxpi7IkMiB4HgFjeE3BzlH_jM=/)
 
 
 
@@ -32,6 +32,14 @@ I discovered the amazing world of algorithmic trading probably about 10 years ag
 
 ## The young "Flash Crash Trader"
 
+## Ethics of *Market Liquidity*
+
+When sh*t hits the fan, all automated systems stop buying also... So where is the liquidity?
+
+## KISS Principle (Keep it Simple _Silly_)
+
+This topic is huge, technical charting is super popular and not fully automated as far as I know. AI analysis of many sources. More Advanced post is coming - to dive deeper into those issues for those interested.
+
 
 ## Notes
 Derivatives in general - are they useful after collapse of 2007/2008? And what about Futures? or Commodities trading?