![]() We have put in a lot of work so far it's time for some answers. insert_all ( "tweets", bundled_tweets ) end map ( tweets, & bundle_tweet / 1 ) TopTweets. def insert_tweets ( tweets ) do bundled_tweets = Enum. With all of this in place, we can finally batch insert all our tweets. Map can be inserted directly into our jsonb field so that we have a As we discussed earlier, the entire tweet_data For each key we pull out the associated piece of dataįrom the tweet_data map. The list of keys in this keyword list are the names of the columns weĭefined in step 1. The process of translating one instance of tweet data into a keyword listĪligning with our tweets table description can be encapsulated in theįollowing function. Translating the retweeted_status data into a boolean is a bit more I cover the details of this conversion in this blog Migration def change do create table ( :tweets ) do add :tweet_id, :bigint, null: false add :author, :varchar, null: false add :text_content, :varchar, null: false add :tweeted_at, :timestamptz, null: false add :retweeted_count, :integer, null: false add :favorited_count, :integer, null: false add :retweeted, :boolean, null: false, default: false add :data, :jsonb, null: false, default: "" ) |> Ecto. Here is the final iteration of our migration: defmodule TopTweets. In the event that we needĪdditional data about a tweet later on, it would be handy to have the entireīody of the API response in the database. Lastly, we are working with a rate-limited API. The addition of an index will go a long way in preventingĭuplicate tweets from getting into our database 2. With this table description, there is nothing to stop us from That sums up the interesting parts of the migration so far. I did just that with the tweeted_at column above. I don't like throwing away timezone information, so I always choose Will refuse to insert the record and complain. Using null: false on all columns ensures that we never leave gaps We expect to always have values for all of our columns when inserting a Sufficient for the purposes of this blog post. Instead chose to let Ecto give me an integer primary key for free. I likely could have used tweet_id as the primary key for this table. Specified in an Ecto migration as the column's data type. We need something that can handle a much larger rangeĪnything that is a valid data type in our backing data store can be Migration def change do create table ( :tweets ) do add :tweet_id, :bigint, null: false add :author, :varchar, null: false add :text_content, :varchar, null: false add :tweeted_at, :timestamptz, null: false add :retweeted_count, :integer, null: false add :favorited_count, :integer, null: false add :retweeted, :boolean, null: false, default: false end end end Here is a first look at an Ecto migration that meets our needs: defmodule TopTweets. Identify each tweet, we will want the id, screen_name of the user, and ToĮnsure that a given tweet is from 2016, we'll want created_at. By including retweet_status we can alsoĭifferentiate between retweets and tweets authored by the given user. We will need favorite_count and retweet_count of each tweet to answer On the other end we have some questions we'd Importing Twitter statuses with ExTwitter has been posted toĪ database schema is often times a middle ground. For the sake of completeness, the code for ![]() The focus of this blog post is on Ecto, so we will only walk through theĭetails of steps 1, 3, and 4. Insert all the imported tweets into the database.Īsk our questions of the Twitter status data using Ecto's schemaless ![]() Import Twitter statuses for a user via the Twitter Here is a roadmap of what we will need toĬreate a database schema for representing and storing tweets in Roadmapĭetermining the most favorited and retweeted tweets of the year is going to With the help of ElixirĪnd Ecto, we can figure that out. If you're an avid Twitter user you may also be wonder what your most popular WhatĪre the goals that you achieved? What are the books that you read? Elixir Top Tweets of 2016 with Elixir and EctoĪs you look ahead to 2017, you may find yourself looking back on 2016.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |