Jekyll2022-02-17T13:43:31-05:00https://dejon.net/feed.xmlDejon in the CloudA website detailing my endeavors tinkering with cloud technology.Dejon WrightCollecting Sports Data Using Node and Sportsradar2022-01-17T00:00:00-05:002022-01-17T00:00:00-05:00https://dejon.net/posts/collecting-sports-data-using-node-and-sportsradar<p>To kickoff generalizing MFD, I’ll start with the data ingestion piece. As I mentioned in my <a href="../2022-01-16-intro-to-myfantasydashboard">MFD intro</a>, I want to make MFD more sport-agnostic while maintaining the existing dashboard layout, but at the individual player/team level. To do this, I’ll need a new sports data provider. Yahoo is great when it comes to fantasy sports data, but only a few sports are a fantasy sport on Yahoo.</p>
<p>To gather data across multiple different sports with a single source provider, I decided to go with <a href="https://www.sportradar.com/">Sportradar</a>. Sportradar is partnered with many of the top professional leagues around the world, so it seemed fitting to start here. They also provide a trial tier that allows for you to access their API free of charge with certain API limitations in place (1 request/second, 1000 requests/month), and it is a single endpoint for all sports.</p>
<p>As I started to dive deeper into the API (which is <a href="https://developer.sportradar.com/docs/read/Home">well documented</a> I may add), I realized I’ll need to use some sort of wrapper for easier interaction. It wasn’t that it was overly difficult to use the API, but there were no provided SDKs for interacting with this data. With Typescript being my latest coding flavor of the month, and being unable to find one that was updated recently or generic enough for what I am planning to do, I decided to write one myself. For anyone interested in getting this data using Python, there is <a href="https://github.com/johnwmillr/SportradarAPIs">solid package</a> already created that does just that, which was helpful as I made the node package.</p>
<p><a href="https://github.com/dwright20/SportradarNode">SportradarNode</a> is a simple node package I threw together to interact with the Sportradar API. The package currently supports retreiving NBA (+ draft), NFL (+ draft), and NCAAMB data, with soccer next up on the list. This package is very much still a work in progress, and is the first package I’ve developed + published, but I plan to continue iterating on it during this MFD journey. You can find the package on the <a href="https://www.npmjs.com/package/sportradar-node">npm registry</a>, and below is an example usage:</p>
<div class="language-typescript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="o">*</span> <span class="k">as</span> <span class="nx">sra</span> <span class="k">from</span> <span class="dl">'</span><span class="s1">sportradar-node</span><span class="dl">'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">nfl</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">sra</span><span class="p">.</span><span class="nx">Nfl</span><span class="p">({</span> <span class="na">apiKey</span><span class="p">:</span> <span class="dl">''</span> <span class="p">});</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">data</span> <span class="p">}</span> <span class="o">=</span> <span class="k">await</span> <span class="nx">nfl</span><span class="p">.</span><span class="nx">getPostgameStandings</span><span class="p">(</span><span class="dl">'</span><span class="s1">2021</span><span class="dl">'</span><span class="p">,</span> <span class="nx">sra</span><span class="p">.</span><span class="nx">NflSeasonType</span><span class="p">.</span><span class="nx">REG</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">playoffTeams</span> <span class="o">=</span> <span class="p">[];</span>
<span class="kd">const</span> <span class="nx">conferences</span> <span class="o">=</span> <span class="nx">data</span><span class="p">.</span><span class="nx">conferences</span> <span class="k">as</span> <span class="p">[</span><span class="kr">any</span><span class="p">];</span>
<span class="nx">conferences</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="nx">conf</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">divisions</span> <span class="o">=</span> <span class="nx">conf</span><span class="p">.</span><span class="nx">divisions</span> <span class="k">as</span> <span class="p">[</span><span class="kr">any</span><span class="p">];</span>
<span class="nx">divisions</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="nx">div</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">teams</span> <span class="o">=</span> <span class="nx">div</span><span class="p">.</span><span class="nx">teams</span> <span class="k">as</span> <span class="p">[</span><span class="kr">any</span><span class="p">];</span>
<span class="nx">teams</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="nx">team</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">team</span><span class="p">.</span><span class="nx">rank</span><span class="p">.</span><span class="nx">clinched</span> <span class="o">!=</span> <span class="dl">'</span><span class="s1">eliminated</span><span class="dl">'</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">playoffTeams</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">team</span><span class="p">.</span><span class="nx">name</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="p">});</span>
<span class="p">});</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">playoffTeams</span><span class="p">);</span>
</code></pre></div></div>Dejon WrightTo kickoff generalizing MFD, I’ll start with the data ingestion piece. As I mentioned in my MFD intro, I want to make MFD more sport-agnostic while maintaining the existing dashboard layout, but at the individual player/team level. To do this, I’ll need a new sports data provider. Yahoo is great when it comes to fantasy sports data, but only a few sports are a fantasy sport on Yahoo.Intro to MyFantasyDashboard2022-01-16T00:00:00-05:002022-01-16T00:00:00-05:00https://dejon.net/posts/intro-to-myfantasydashboard<p>MyFantasyDashboard (MFD) came from the dissatisfaction I got from looking at tables of data to see how my fantasy football team was doing. Since I’m a visual learner, I would much rather look at a graphic that makes it clear and obvious how I’m doing. With this annoyance in mind, and being the techie that I am, I figured ‘maybe I can just make something like this myself.’ And that’s where the MFD personal project spiral began.</p>
<p>There were 2 main things I felt were a requirement for this project as I started:</p>
<ol>
<li>Data had to come from Yahoo
<ul>
<li>Since I was playing in Yahoo fantasy leagues, and Yahoo makes fantasy data readily available for users, this was a hard requirement (and a no brainer 🤷🏾♂️)</li>
</ul>
</li>
<li>Needs to be hosted <em>somewhere</em>
<ul>
<li>Sharing this with others was something I wanted to have the option of doing at some point, and texting someone a link to click vs a file - to download, run, configure, blah, blah - was, again, a no brainer 🤷🏾♂️</li>
</ul>
</li>
</ol>
<p>With these requirements in mind, 3 questions needed to be answered:</p>
<ol>
<li>How do I get access to a user’s specific fantasy data?
<ul>
<li>A. Yahoo provides a <a href="https://developer.yahoo.com/oauth2/guide/">well documented guide</a> for using OAuth 2.0 to access a user’s fantasy data, and also provides a useful <a href="https://developer.yahoo.com/oauth2/guide/flows_authcode/">authorization code flow diagram</a> to aid in the implementation.</li>
</ul>
</li>
<li>How do I want the data to be visualized?
<ul>
<li>A. A lot of brainstorming went into this (probably too much… 😅). Ultimately, I decided on a dashboard-esq view that focuses on your team vs your opponent’s team, has additional graphs to show historic data, and shows league wide metrics - an unaccomplished stretch goal. Here is the sticky note “dashboard” I used to map out how the actual dashboard would look: <img src="/assets/images/MFD/MFD-sticky-notes.jpg" alt="MFD layout sticky notes" /></li>
</ul>
</li>
<li>How slow is this going to be??
<ul>
<li>Wasn’t sure. Figured it would probably be slow if it was going to be making calls to Yahoo each time I wanted to view the player data. But I also needed to get my hands dirty with the Yahoo API to understand all of its nuances.</li>
</ul>
</li>
</ol>
<p>Based on these requirements and my (partially) answered questions, I started building. The result was something I was <em>okay</em> with - it was simple, showed what I wanted to see, and I could send a link to others and have them use it too 🥳</p>
<p>Sample MFD graphs:</p>
<figure class="half">
<a href="/assets/images/MFD/web-pie.jpg"><img src="/assets/images/MFD/web-pie.jpg" /></a>
<a href="/assets/images/MFD/web-stacked-bar-total.jpg"><img src="/assets/images/MFD/web-stacked-bar-total.jpg" /></a>
<figcaption>Web views - generated with React</figcaption>
</figure>
<figure class="third">
<a href="/assets/images/MFD/web-mobile-line.jpg"><img src="/assets/images/MFD/web-mobile-line.jpg" /></a>
<a href="/assets/images/MFD/ios-dashboard-pie.jpg"><img src="/assets/images/MFD/ios-dashboard-pie.jpg" /></a>
<a href="/assets/images/MFD/ios-dashboard-bar.jpg"><img src="/assets/images/MFD/ios-dashboard-bar.jpg" /></a>
<figcaption>Mobile views - 1st generated with React, 2nd + 3rd generated with Swift</figcaption>
</figure>
<p>There were a few things I didn’t like though:</p>
<ul>
<li>Data to screen speed
<ul>
<li>It took what felt like 2/3 mins from the time I clicked refresh on the dashboard to when the new data was updated. This is fine if you want a one-off snapshot view of your team, but having to wait this long while trying to have a ‘real-time’ view of your team is impractical.</li>
</ul>
</li>
<li>Repeated processing
<ul>
<li>There were overlaps in players between my teams and as I switched from viewing one team’s dashboard to the next, the process of collecting those players’ data would be repeated. Since all the data was coming directly from Yahoo and there was nowhere to store the players’ data, the work was being repeated.</li>
</ul>
</li>
<li>Lack of persistence for analytics
<ul>
<li>I am (probably) what the AI/ML experts would call a ‘script-kiddie’ AI/ML enthusiast. With that being said, I wanted to be able to run some of my own modeling on my fantasy teams. With the current flow of web app -> API -> processing Lambda -> Yahoo and back, there was no way I could run analyses on the data being collected.</li>
</ul>
</li>
</ul>
<p>To address these concerns, I implemented data storage on the backend. The idea was to only follow the flow mentioned above when getting details about the specific team - record, opponent, current roster, waiver order, etc. - and, when that data is collected, push the player keys into a table as the primary key to maintain uniqueness. There would then be a scheduled event that would kickoff a flow to collect stats on each player in the table and store the data in a different table. To collect the players’ stats, the data would then come from the players’ stats table instead of Yahoo.</p>
<p>Now that fantasy football season has ended (I didn’t even finish top4 to make the dance 🙁), I have set my sights on doing a similar pattern for other sports and in a more generic fashion. The big question I’m tackling next: can I display the same type of information I did for a fantasy team, but at the individual player level (Ex. Tom Brady) or individual team level (Ex. Chicago Bears - yes, they’re my favorite team 😅)?</p>Dejon WrightMyFantasyDashboard (MFD) came from the dissatisfaction I got from looking at tables of data to see how my fantasy football team was doing. Since I’m a visual learner, I would much rather look at a graphic that makes it clear and obvious how I’m doing. With this annoyance in mind, and being the techie that I am, I figured ‘maybe I can just make something like this myself.’ And that’s where the MFD personal project spiral began.Welcome to Dejon in the Cloud!2022-01-01T00:00:00-05:002022-01-01T00:00:00-05:00https://dejon.net/posts/welcome-to-dejon-in-the-cloud<p>Hello, and welcome to my personal website and blog! I am a Solutions Architect at Amazon Web Services (AWS). I created Dejon in the Cloud as a platform to write about my personal techy projects and post resources I’ve found helpful during development. I understand the frustration of trying to fix bugs and errors or just trying to find a starting point - hence, <em>trying not to tilt</em> - so my goal with this site is for my posts to be a resource to other developers, as I’ve found such posts to be extremely helpful for my projects.</p>
<p>Here are a few projects I’m currently working on and will be writing about soon:</p>
<h3 id="myfantasydashboard">MyFantasyDashboard</h3>
<blockquote>
<p>An app experimenting with sports data using a serverless architecture. Goal is to have a system that tracks and displays sports data per player, team, or an individual’s Yahoo fantasy team.</p>
</blockquote>
<h3 id="barbot">BarBot</h3>
<blockquote>
<p>A project for creating an IoT beverage dispenser that can create drinks based on the liquids provided. Will be app controlled and have recommendation logic integration.</p>
</blockquote>
<h3 id="mdrobot">mdRobot</h3>
<blockquote>
<p>Tinkering with the iRobot Create 2. Goal is to get hands-on with robotics and the integration with AWS services.</p>
</blockquote>Dejon WrightHello, and welcome to my personal website and blog! I am a Solutions Architect at Amazon Web Services (AWS). I created Dejon in the Cloud as a platform to write about my personal techy projects and post resources I’ve found helpful during development. I understand the frustration of trying to fix bugs and errors or just trying to find a starting point - hence, trying not to tilt - so my goal with this site is for my posts to be a resource to other developers, as I’ve found such posts to be extremely helpful for my projects.