Artwork

Contenido proporcionado por Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Todo el contenido del podcast, incluidos episodios, gráficos y descripciones de podcast, lo carga y proporciona directamente Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® o su socio de plataforma de podcast. Si cree que alguien está utilizando su trabajo protegido por derechos de autor sin su permiso, puede seguir el proceso descrito aquí https://es.player.fm/legal.
Player FM : aplicación de podcast
¡Desconecta con la aplicación Player FM !

What can Apache Kafka Developers learn from Online Gaming?

55:32
 
Compartir
 

Manage episode 424666711 series 2510642
Contenido proporcionado por Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Todo el contenido del podcast, incluidos episodios, gráficos y descripciones de podcast, lo carga y proporciona directamente Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® o su socio de plataforma de podcast. Si cree que alguien está utilizando su trabajo protegido por derechos de autor sin su permiso, puede seguir el proceso descrito aquí https://es.player.fm/legal.

What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern online games are complex, distributed real-time data systems with decades of innovative techniques to teach us.
In this episode, Ben talks with Kris about integrating gaming concepts with Apache Kafka®. Using Kafka’s state management stream processing, Ben has built systems that can handle real-time event processing at a massive scale, including interesting approaches to conflict resolution and collaboration.
Building latency into a system is one way to mask data processing time. Ben says that you can efficiently hide latency issues and prioritize performance improvements by setting an initial target and then optimizing from there. If you measure before optimizing, you can add an extra layer to manage user expectations better. Tricks like adding a visual progress bar give the appearance of progress but actually hide latency and improve the overall user experience.
To effectively handle challenging activities, like resolving conflicts and atomic edits, Ben suggests “slicing” (or nano batching) to break down tasks into small, related chunks. Slicing allows each task to be evaluated separately, thus producing timely outcomes that resolve potential background conflicts without the user knowing.
Ben also explains how he uses pooling to make collaboration seamless. Pooling is a process that links open requests with potential matches. Similar to booking seats on an airplane, seats are assigned when requests are made. As these types of connections are handled through a Kafka event stream, the initial open requests are eventually fulfilled when seats become available.
According to Ben, real-world tools that facilitate collaboration (such as Google Docs and Slack) work similarly. Just like multi-player gaming systems, multiple users can comment or chat in real-time and users perceive instant responses because of the techniques ported over from the gaming world.
As Ben sees it, the proliferation of these types of concepts across disciplines will also benefit a more significant number of collaborative systems. Despite being long established for gamers, these patterns can be implemented in more business applications to improve the user experience significantly.
EPISODE LINKS

  continue reading

Capíttulos

1. Intro (00:00:00)

2. Building real-time inspection machines (00:05:29)

3. Going Multiplayer with Kafka (00:07:14)

4. What does a real-time experience look like in gaming? (00:14:35)

5. Broken message offsets vs. dead letter queues (00:17:05)

6. The distributed transaction problem (00:19:15)

7. Solving for conflicting events with nano-batching (00:25:02)

8. Pooling in real-time systems (00:31:56)

9. Optimizing for latency (00:38:29)

10. Using column modeling to improve performance (00:44:08)

11. Principles for stream-processing (00:50:49)

12. It's a wrap! (00:52:46)

265 episodios

Artwork
iconCompartir
 
Manage episode 424666711 series 2510642
Contenido proporcionado por Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Todo el contenido del podcast, incluidos episodios, gráficos y descripciones de podcast, lo carga y proporciona directamente Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® o su socio de plataforma de podcast. Si cree que alguien está utilizando su trabajo protegido por derechos de autor sin su permiso, puede seguir el proceso descrito aquí https://es.player.fm/legal.

What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern online games are complex, distributed real-time data systems with decades of innovative techniques to teach us.
In this episode, Ben talks with Kris about integrating gaming concepts with Apache Kafka®. Using Kafka’s state management stream processing, Ben has built systems that can handle real-time event processing at a massive scale, including interesting approaches to conflict resolution and collaboration.
Building latency into a system is one way to mask data processing time. Ben says that you can efficiently hide latency issues and prioritize performance improvements by setting an initial target and then optimizing from there. If you measure before optimizing, you can add an extra layer to manage user expectations better. Tricks like adding a visual progress bar give the appearance of progress but actually hide latency and improve the overall user experience.
To effectively handle challenging activities, like resolving conflicts and atomic edits, Ben suggests “slicing” (or nano batching) to break down tasks into small, related chunks. Slicing allows each task to be evaluated separately, thus producing timely outcomes that resolve potential background conflicts without the user knowing.
Ben also explains how he uses pooling to make collaboration seamless. Pooling is a process that links open requests with potential matches. Similar to booking seats on an airplane, seats are assigned when requests are made. As these types of connections are handled through a Kafka event stream, the initial open requests are eventually fulfilled when seats become available.
According to Ben, real-world tools that facilitate collaboration (such as Google Docs and Slack) work similarly. Just like multi-player gaming systems, multiple users can comment or chat in real-time and users perceive instant responses because of the techniques ported over from the gaming world.
As Ben sees it, the proliferation of these types of concepts across disciplines will also benefit a more significant number of collaborative systems. Despite being long established for gamers, these patterns can be implemented in more business applications to improve the user experience significantly.
EPISODE LINKS

  continue reading

Capíttulos

1. Intro (00:00:00)

2. Building real-time inspection machines (00:05:29)

3. Going Multiplayer with Kafka (00:07:14)

4. What does a real-time experience look like in gaming? (00:14:35)

5. Broken message offsets vs. dead letter queues (00:17:05)

6. The distributed transaction problem (00:19:15)

7. Solving for conflicting events with nano-batching (00:25:02)

8. Pooling in real-time systems (00:31:56)

9. Optimizing for latency (00:38:29)

10. Using column modeling to improve performance (00:44:08)

11. Principles for stream-processing (00:50:49)

12. It's a wrap! (00:52:46)

265 episodios

Tất cả các tập

×
 
Loading …

Bienvenido a Player FM!

Player FM está escaneando la web en busca de podcasts de alta calidad para que los disfrutes en este momento. Es la mejor aplicación de podcast y funciona en Android, iPhone y la web. Regístrate para sincronizar suscripciones a través de dispositivos.

 

Guia de referencia rapida