diff --git a/ndc-ldn-2024/mono-micro.jpg b/ndc-ldn-2024/mono-micro.jpg new file mode 100644 index 0000000..6b23345 Binary files /dev/null and b/ndc-ldn-2024/mono-micro.jpg differ diff --git a/ndc-ldn-2024/ndc.jpg b/ndc-ldn-2024/ndc.jpg new file mode 100644 index 0000000..1061532 Binary files /dev/null and b/ndc-ldn-2024/ndc.jpg differ diff --git a/ndc-ldn-2024/readme.md b/ndc-ldn-2024/readme.md index 7b4f479..b0ffc43 100644 --- a/ndc-ldn-2024/readme.md +++ b/ndc-ldn-2024/readme.md @@ -1,6 +1,8 @@ # NDC London 2024 +![ndc](ndc.jpg) + ## Wednesday, 2024-01-31 @@ -51,44 +53,612 @@ https://ndclondon.com/agenda/understand-the-next-phase-of-web-development/04mxuc - AlRa: 04, 11 +## Ratings + +0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 + +1:xxxx - initial review +2:xxx - adjusted review 2 +n: xxxx - adjusted n + ## Keynote: Debug your thinking +1: 21 + +- Single sparkler age šŸ˜‚ +- building on a foundation that wasnā€™t designed for the scale +- sales / product selling something before it is ready / poc work +- making the right architecture choices at the start +- doing everything right from the start - worry of running out of budget +- you think you are doing everything right, but have you considered all the variables? +- no such thing as right architectureā€¦ no perfect language.. +- what decisions have you made? +- what was impact? +- what negatives? +- what problems trying to solve? +- long term or short term? +- how many options considered? +- assumptions on context? +- how not overly bias on my ā€œchosenā€ solution? +- how do I debug my thoughts? +- critical thinking +- debug your thinking +- the trap: working in solution modeā€¦ +- ohh i have this problem, can you fix it for meā€¦ +- this is dangerous, bias into jumping into solution mode +- we are incentivised to jump into this. We need time to understand the problem. +- we might be trying to solve the wrong problem to begin with +- we need to slow downā€¦ +- is it actually a problem +- people often bring us ā€œa solutionā€ +- we must always dig deeper to try to understand what is going on +- always keep asking ā€œwhyā€ +- 5 whysā€¦ šŸ™„ +- use it for framing initial analysis +- write it down +- writing is natures way of telling you how sloppy your thinking is šŸ˜‚šŸ˜‚šŸ˜‚ +- got to make your thinking concise and clear +- framing problem: +- writing down +- avoid tech terms +- no solutions! +- make it measurable +- then ask if this is the right problem +- the elevator problem +- change from elevator is to slow, to, the wait time is to annoying +- added mirrors šŸ˜†šŸ˜†šŸ˜† people stopped complaining +- reframing a problem +- we are here to solve business problems +- we should call out if there is a solution that involves no technical involvement at all +- ask: +- is this statement even true? +- who is affected by this? +- is there solution bias? +- what happened before this problem was occurring? +- hidden influences? +- sub optimal solution to the right problem is so much better than a perfect solution to the wrong problem +- BIAS to speed JFDI +- why sql server used so much to store json šŸ˜‚ +- we are techies we want to use cool stuff +- children are naturally out of the box thinkers +- convergent and divergent thinking +- other peopleā€™s suggestions might be outside of our own box +- we feel safe in our own box of thinking +- curiosity over judgement +- hmm thatā€™s an interesting idea, can you walk me though your thinking process on it? +- get different perspectives +- cognitive bias - itā€™s a feature not a bug šŸ˜‚ +- it give swift thinking - useful at times in real life +- but donā€™t need that quick reaction at work +- most common biases: +- false causalityā€¦. Are those events all connected?ā€¦ +- HIPPO effect.. highest paid persons opinion šŸ˜‚šŸ˜‚ +- authority bias +- confirmation bias +- assumptionā€¦ something we believe so much we donā€™t even question it.. +- why do I actually believe this? +- how to catch your biases: +- mindfulnessā€¦ building Lego is mindful +- invite a devils advocate +- their job to involve in observing and catching the bias assumptions. Assign someone to the role and rotate it +- we want your feedback - rfc. Scale it big and you donā€™t have to wait for everyone. !!!!!!!!!!!!!!!! +- take aways; +- slow down +- write it down +- work together and collaborate +- +I + ## Failing Successfully +1:5 + +CANCELLED. Instead https://ndclondon.com/agenda/real-time-connected-apps-with-net-maui-blazor-and-signalr/0179973be3cc + + + +- ms employee +- signal r - started with basic example +- Maui +- evolution of xamarin forms +- wrapper on iOS and andoird Mac and windows apis +- blazor. Run it all in c# +- forget needing to know js.. +- blazor united - new in 8 - hybrid of server and web assembly. !!!!!! +- maui and blazorā€¦: +- = blazor hybrid. +- dotnetpodcasts.com - a cool example app. Is on GitHub. + ## You Keep Using That Word: Asynchronous And Interprocess Comms +1: 13 +![async](you-keep-using-that-word.jpg) + +- boxes and arrows šŸ˜‚ +- meaning of words +- this talk is not about the boxes, but about the arrows +- microservice architecture šŸ˜‚ +- arrow =>dependancy +- some sort of logical dependancy +- EDA - itā€™s all about facts. Something has happened. +- an inversion to o request response model. There is the dependancy but itā€™s implemented the other way round +- donā€™t get stuck in the con game (v3) +- use term eventual convergence instead of eventual consistency +- The reactive manifesto (2014 v2) +- problem with blocking calls - sum latency +- temporal decoupling +- intermediary between A and B eg message broker +- need to trust your broker +- stateless stateless stateless +- making requests idempotent makes retryā€™s safe +- SET - the word with the most meanings in the OED. 430 different meaning. RUN, will have 645 in next draft. +- derive meaning of a word from its context +- ā€œin the context and of this paperā€¦..ā€ context is important šŸ˜Š +- socialtechnical systems +- Tower of Babel old test +- Sam Newman - building microservices, oreilly books +- + +## Lunch + +- does pressing lift buttons with a non body part increase chance of lift coming? +- is static from body just lighting up the button led?ā€¦ + ## Actionable Observability +1:8 + +- observability: understanding what is happening inside software systems to debug a problem, from telemetry emitted by system +- utilising telemetry with goal of improving application reliability +- identify, debug, address tech issues +- incident management cycle +- role of automation.. reduce complexity, manage cognitive load +- collect ->identify->debug->ā€¦. +- automate identity otherwise sat there looking at dashboards. Be proactive. +- infra metrics, service metrics, app metrics, user experience metrics +- define user journey, define metric, define thresholds. Eg: +- message send request submitted. => time to result of send received => a thresholdā€¦ +- coming up with thresholds will often be iterative +- service level objectives: +- prioritise user journeys, focus on most important +- ā€œNorth Starā€ metrics +- donā€™t let SLOs become overwhelmingly huge +- Service Level Indicator - reliability is different to availability. SLI should represent reliability. Eg very high latency could be counted equivalent to offline. +- setting error budget +- reliability 0-100% score +- alerting strategy: +- consumption of budget vs rate of change of budget (burn rate) +- automation not about replacing us, itā€™s about enabling us +- +- + +## Snack bar + +![snack](snacks.jpg) + ## Architecture Modernization: Aligning Software, Strategy, and Structure +1:21 / 34 + +- nick tune- architecture modernisation - book +- ran the workshop sessions on ddd that JB attended +- teams deploying multiple times a day +- need to see it early enough that you need to modernise +- open table stopped all features and modernised for 8 monthsā€¦ not always advocating that approach +- doing nothing- how long until irrelevant or fail? +- even if make good arch decisions, your assumptions your decisions are based on will change over time +- result: still end up with messy architectureā€¦ +- arch modernisation -> convert disadvantage to advantage +- talk overview: +- architecting for flow +- tools +- kickstarting and enabling arch modernisation +- rewriting / using new tech, doesnā€™t fix all the problems. Itā€™s deeper than that. It enables you to then be able to fix actual problem +- make changes / features fit inside one subdomain +- empower teams +- software decoupled +- always will have the question of should we do modernisation work or new feature work +- time to listen - indi young - book +- need the business objective to support the modernisation decisions +- impact mapping +- wardley mapping +- make decisions based on how component is evolving, not its current state +- ask how quickly each component is evolving +- Event storming. Big picture and process modelling +- getting a true wide-perspective picture +- domains and subdomains +- grouping and transition points +- modernisation strategy selector.. +- how to maximise roi on modernisation +- platform modernisation and also, +- product, domain, software modernisation +- codescene tool +- keeping momentum after event storming workshops +- architecture modernisation enabling team AMET. Donā€™t own a part of system, but help them achieve the change. +- team for range of skills not just archā€™s. +- donā€™t get lost in all upfront planning +- make a decision on when first change will be seen. Aim a target of 3 - 6 months? +- patterns for migrations. Strangler fig. DDD bubble pattern. Danger of strangler fig. +- + ## The definitive deep dive into the .git folder +1:13 + +- .git that folder is all you need +- objects - all git zlib compressed. Files created on commit. Files are named by hash of commit +- all the slides are on his GitHub +- the commit reference another object id node +- tree node lists files and permissions +- and is double zlib compressed +- the tree references blobs +- git cat-file +- git add puts the file in as a blob - itā€™s staged. +- the commit creates the commit object and links it to the blobs via tree object +- folders nested are represented by another tree node +- git commit with -amend, this changed the file. But history is still stored. +- branches - refs folders +- git remotes can also be relative local file system pathsā€¦ +- hooks folder, great for automation +- although hooks canā€™t be committedā€¦ so look at npm githooks which use symlinks +Git config files - alias section + + ## Supercharge your team by building clarity, not consensus + +CANCELLED + +instead watch this: + +https://ndclondon.com/agenda/ada-lovelace-and-the-very-first-computer-program-0bv2/0s6o9bizrgp + + ## So You Want to Build An Event Driven System? +1:21 + +- yuval Noah harari- sapiens - book: +- about stories being so important +- EDA is a communication pattern +- modelling the integration +- James Eastham - aws architect +- built EDA in the past, and many have failed +- learn more from failure than success +- example is that we are devs at plant based pizza, system currently request response. +- order processing have dependency on everything, kitchen, delivery etc +- aim for high cohesion, and low coupling +- an appropriate level of coupling depends on the level of control you have over the endpoints. +- Martin fowlers talk on eda: +- what is eda - reversing dependencies +- what is an event.. An immutable fact. Something that has happened in the past. It cannot be changed. +- event driven system vs event based system: +- eds- itā€™s driven by business events eg: PizzaBoxed, DeliveryNotificationSent. +- tell the story of your business through events +- events are first class citizens: events designed even before api.: Event First Design. +- notification event. Lightweight. Anaemic. Often leads to an api call to the sender. Can lead to fire, as far to many calls back to originator. The producer of events should be unaware of downstream consumers. +- event carried state transfer - a fat event. More in there for downstream consumers can use. +- producer, consumer and event broker (bus, queue etc) - pub sub. +- always put a queue in front of compute. Eventbus ==>>> my domain (my queue => my compute). Protect your self from overload. +- go do event storming - to understand what the business events are +- governance for controlling eventsā€¦ stop the unknowns breaking. +- put out a public RFC before changing. +- payload. Split to data and meta. Meta should contain ids, event type, version. VERSION NUMBER. Depreciated flag? Sunset dates. +- fat vs sparse. Always - it depends. +- enriched pattern +- use eventual convergence over term eventual consistency. Embracing this is the EDA trade off. +- EDA dealing with http request response. Wrap events into a way to make call, and the publish event when finished, with a queue. +- meta data to contain trace ids. Previous event that triggered it. +- boundaries. Going to need request response somewhere at the edge. Really thin synchronous wrapper that dumps onto event bus - posts. +- but for gets.. separation of command events and queries. +- processing commands synchronously. Eg send an email. +- where should the responsibility lay. If truely event, eg email service listening to order created. Using queue for async commands. +- load on cqrs. +- transactional outbox pattern. Storing data in service before publishing it to the service. +- change data capture - stream output, publish events from the back of it. Using dynamodb streams. publish events. +- redid cache for query. + + +Question: integration event bus, fire and forget, or persistent and support event replay ability? + +It depends on the use case. Does it need to get there? Then if persistent horizontal client scaling is hard. Or fifo queue. Ugh. + +Question: how to deal with the common question of: now everything is dependant on a single event bus. Worry? + + + + ## Distribu-ready with the Modular Monolith +1:13 +![micro mod](mono-micro.jpg) + +https://ndclondon.com/agenda/distribu-ready-with-the-modular-monolith-0bk5/0tqfug96b0i + + +https://ndclondon.com/speakers/layla-porter + + + +- complexity is your enemy. Anyone can make something complicated. Itā€™s hard to keep it simple.. Richard Branson +- monolith. Single executable. Some benefits. But problems. Simple deploy. +- hard to scale and maintain. +- microservice architecture. Divide into lots of small independently deployable services. +- what needs? Want to scale. What are options: +- go through code or infra and scale vertical / horizontal. +- or distributed system. +- the cost of distributed is simplicity. +- distro monolith is the worse place to be +- shared db is the evil of distro monoliths. +- DM: hard bits of complexity. But not of the independence benefits. None of the monolith ease of debug. +- DDD. Lose coupling and high cohesion. +- the more pleasant system to get thereā€¦ +- the modular monolith. +- mono to modular mono (not distro mono) to microservice. +- the kitkat pattern. +- how to decompose the mono app. +- decomposing the database. Code first or db first. START db separation first. Then move onto code separationā€¦ +- be pragmatic, not dogmatic. +- soā€¦. The how?: +- how to ensure completely independent? +- how to communicate?: +- external not much change. +- internal inter proc comms: +- mediator pattern. +- messaging pattern. +- thin and fat content payloads +- how to enforce architectureā€¦ code review. +- .net aspire https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview + +Question: this focused on mono to micro. What happens if already in the distro mono hole? + +A: pray šŸ˜‚ communicate why itā€™s a problem. Need people to see why itā€™s a bad place to be. + + ## Better Code Reviews FTW! +1: 13 + +- why do code reviews? +- finding defects least of concerns? +- it should focus on sharing knowledge and sharing patterns +- sharing knowledge +- shared ownership of code in the squad. +- on merge itā€™s going from my code to our code +- how to do code reviews like a human +- be open for feedback +- appreciate what people do right, not brining down on the what they do wrong. +- award ties in discussions to the reviewer.! (The reviewer has a fresh set of eyes..) or at least consider this. +- review your own PR first +- have an absolute style guide. +- PR review should be a higher priority than anything else at all. Help someone to finishing something is better than starting something new. +- developers donā€™t have much patience. From a given list will pick the low hanging fruit and ignore the complex points +- regular little is the way forward. Through PR aim to get D grade code to C grade, not A grade. Next time itā€™s touched it can go from C to B/A. Or do a separate extra or to improve that. +- suggest code changes rather than just saying something is bad / wrong. Teach. +- framing feedback as questions +- if your arguing in code reviews, probably doing it wrong. + + ## How to get a grip on your microservices system using a service-mesh +- this is heavy on k8s +- what is service mesh and why use it +- observability +- chaos engineering +- service mesh almost always use sidecar pattern +- ambient mesh coming soon, does it without another container +- sidecar runs alongside actual container, in the same pod +- istio as a mesh provider +- kiali very cool graph views ok k8s injections and traffic etc between them +- service mesh - power comes from mirroring requests to new versions of a service. Ie test a new version with real live data (obv any transactions etc have to be separate db etc) +- chaos engineering, injection faults, including network level style failures + + + ## Let's build the worst Event Sourcing system! +1: 21/34 + +![worst es](worst-es.jpg) + +- project franz +- worst es system ever +- event store db +- Kafka for event storing or event bridge +- idempotency +- event ordering +- the queue we are standing in is always the slowest šŸ˜‚ +- Kafka topic and partition split. Topic is logical, partition is physical. +- event disordering šŸ˜‚ +- process managers. +- the confusion between event sourcing and event streaming +- optimistic concurrency +- strong consistency +- Kafka canā€™t do the optimist concurrency +- basically donā€™t use Kafka for event sourcing. +- Kafka is good for what it should be used for. +- Kafka is for event streaming. +- event sourcing is about recording and storing business events. +- event streaming is about moving data from one place to another. +- cap theorem - consistency, availability, partitioning - canā€™t have all 3 +- outbox pattern, store state and output message at same time. Both or neither +- CRAP theorem - managers wanting 100% of c an and pā€¦ +- passive aggressive events +- eg the bin is full, the dishwasher has finished šŸ˜‚šŸ˜‚šŸ˜‚šŸ˜‚ +- stream design. Event sourcing is still just storage. +- events on the inside and events on the outside. +- domain and integration events. +- important to have the distinction. +- keep granular events in your domain, in your domains own event store. +- long living streams. More events a stream, iops goes up +- snapshots. Every event? Some period of time? Every x events? +- hope driven design HDD +- snaps shots are a cacheā€¦ +- closing the book. Eg rather than account, accounting month. Stream restarts each month. +- a bad system will beat a good person every time. +- fear uncertainty and doubt +- Peter principle +- running with an empty barrow + + ## Itā€™s time to rebuild DevOps. +- breaking the mould +- may be a triggering talk for some šŸ˜‚ +- gene kim devops handbook, book +- Chris reed. Unless your pretty code is in production, youā€™re just wasting time. +- 2019 there were 80 devops days +- already 60 this year +- devops eating up the world +- solving comms between devs and ops +- Etsy - infra / devops done well +- key problems: +- paper cuts. All the little things that are a bit hard. Tool after tool to fix crap they comes before. +- collaboration. Pull requests / git, increase barriers to collaboration. +- source of truth. The actual infra, or the source control?ā€¦ +- Rick Rubin. Book / quote. Average is nothing to aspire to. +- system initiative. Utilises digital twins. +- local stack + + ## CS Fundamentals: Why SSL and SSH are Secure +Alternate instead + +https://ndclondon.com/agenda/tracking-aircraft-with-streams-software-defined-radio-075j/0m0z3hlqpf2 + +- redis stack for streaming and event state + + ## NDC Party - Conference reception +- šŸøšŸøšŸø +- šŸŗšŸŗšŸŗ + ## Keynote: Transformers: The Rise of ChatGPT +13 + +![rise chat gpt](rise-gpt.jpg) + +- kesha Williams +- transformer architecture patterns +- GPT - stands for: +- generative pretrained transformer +- talk: +- origins +- deep dive +- real world +- more than meet eye +- commanding power +- autobots roll out. +- Natural language processing +- nlp helps machines understand our language +- nlp, seri Alexa +- gen ai is for nlp what the all spark was for Optimus prime šŸ˜‚šŸ˜‚ +- GenAI, creates new content +- powered by large foundation models +- top layer, AI, next is ML, then Deep Learning, under that GenAI +- FM FOUNDATION MODEL +- fm powering chat GPT is called GPT +- One fm can replace many prior gen models +- pre train, and adapt & fine tune +- chat gpt is fine tuned gpt +- input processing, understand context, generation +- 2018 transformer architecture +- 2018 gpt1, text generation summarise and translate +- 2019 gpt2 gpt1 + qa +- 2020 gpt3 gpt2 + code gen + creative writing +- 2022 gpt3.5 gpt3 + fine tuned for conversation +- 2023 gpt4 multi modal image etc +- message KW on linked in for free access to her linked in learning courses +- cloud career journeys - book +- + ## Software Reimagined: The Power of Vertical Slice Architecture +Missed due to Long chat with Oskar Dudycz re event sourcing and event db + +- closing the book strategies +- right to be forgotten, encryption payloads, multiple rolling keys etc +- persistent subs +- event store cloud offering +- iot usage +- todo read his esdb blog post on closing book + other links heā€™s going to send through +- https://www.eventstore.com/blog/keep-your-streams-short-temporal-modelling-for-fast-reads-and-optimal-data-retention + +Went to end of whatā€™s new in net 8 + +- interceptors +- debuggerhidden attribute šŸ˜‚šŸ˜‚šŸ˜‚šŸ˜‚ +- span, stack, hence not in async await. MemoryT uses heap +- collection to expressions +- optional parameters on lambdas, niiiice +- alias any type +- filip ekberg +- filip@ekberg.dev +- + ## How GitHub delivers GitHub using GitHub +13 / 21 + +- April Edwards +- been at GitHub for 11 months +- at ms before +- 3 week sprints. But it is set per feature team, vertical slices. +- at GitHub, all non tech staff use GitHub for their work, hr, sales, all of them +- no local code +- all GitHub GitHub dev is using code spaces +- epics, stories, tasks +- epics mgmt quarterly +- extensive use of feature flags +- docsets, collection of repos +- code spaces +- use of devcontainer for pre setting extensions for vs code workspaces +- + ## Keep your nose out of it. Denying yourself access to production +1: 5 +- Glenn +- why block yourself: +- blamability. You canā€™t touch it so canā€™t be blamed. Arse cover. +- criminal users at work +- criminal external +- limit attack surface +- explicit and auditable +- big shop Vs small shop, governance +- need to keep some agility. Deploying. +- zero trust. Assume breach. +- least privileged access +- application segmation, fits with microservices. Only have access to your part. +- azure managed identities, only can ever be assigned to a service. +- push logs out of prod that donā€™t have PII. +- just in time access +- azure PIM, request access for a set amount of time for certain resources, with reason specified and approval process. +- immutable ledgers for audit logs +- make a tool for it šŸ˜Š + ## Co-Create: Creating Better Together -## How JavaScript Happened: A Short History of Programming Languages \ No newline at end of file +- todo: watch online + +## How JavaScript Happened: A Short History of Programming Languages + +- todo: watch online + +# general notes + +- implement org or arch wide RFC process +- notify AMET team +- notify set target for first ā€œchangeā€ in modernisation process +- in maturity doc - eda for comms. Micro arch for compute. +- whys. +- what is our distro-ready state - ie the point we are ready to separate. + +## es next step session + +- map out sub domain boundaries +- explore how the db can be separated +- map out what are the contract for events. Utilising dynamo db change stream. \ No newline at end of file diff --git a/ndc-ldn-2024/rise-gpt.jpg b/ndc-ldn-2024/rise-gpt.jpg new file mode 100644 index 0000000..7d610ff Binary files /dev/null and b/ndc-ldn-2024/rise-gpt.jpg differ diff --git a/ndc-ldn-2024/snacks.jpg b/ndc-ldn-2024/snacks.jpg new file mode 100644 index 0000000..84ae4f0 Binary files /dev/null and b/ndc-ldn-2024/snacks.jpg differ diff --git a/ndc-ldn-2024/worst-es.jpg b/ndc-ldn-2024/worst-es.jpg new file mode 100644 index 0000000..8c4d6c4 Binary files /dev/null and b/ndc-ldn-2024/worst-es.jpg differ diff --git a/ndc-ldn-2024/you-keep-using-that-word.jpg b/ndc-ldn-2024/you-keep-using-that-word.jpg new file mode 100644 index 0000000..3a1de26 Binary files /dev/null and b/ndc-ldn-2024/you-keep-using-that-word.jpg differ