How to Download All Your DEV Articles in Markdown Format

January 3, 2020

Update: Looks like the dev API has changed. I added a new params option to get max 1000 articles.

We can agree that DEV is the best site for devs in the world. What if you want to cross-post your dev articles into your own personal blog site, but too lazy to copy-paste each post? 😴

I used to copy paste every single article I wrote into my personal site, but quickly found it tedious. I created a script to easily generate DEV articles.

There are two ways to do this: work from the script below or clone the Github repo.

The script

Here is main script. It is pretty short. It uses axios to fetch the data and dotenv to store secrets.

require('dotenv').config()
const axios = require('axios')
const fs = require('fs')
const path = require('path')
const URL = 'https://dev.to/api/articles/me/published'

const options = {
  headers: { 'api-key': process.env.DEV_KEY },
  params: { 'per_page' : 1000 }
}

const contentDir = '/content/'
const contentPath = __dirname + contentDir

axios
  .get(URL, options)
  .then(response => {
    response.data.forEach(article => {
      const normalizedTitle = article.title
        .replace(/\s+/g, '-')
        .replace(/[^a-zA-Z0-9-]/g, '')
        .toLowerCase()
      const normalizedPublishedAt = article.published_at.split('T')[0]
      const fileName = `${normalizedPublishedAt}_${normalizedTitle}.md`

      console.log(`${contentDir}${fileName} created!`)
      fs.writeFileSync(
        `${contentPath}${fileName}`,
        article.body_markdown,
        'utf-8'
      )
    })
    console.log('*** dev.to data successfully fetched ***')
  })
  .catch(error => {
    console.log('ERROR: ', error)
  })

DEV API makes it very easy to do it. The response of the API request returns an array of all articles you've written.

I extracted the markdown content, renamed the file with YYYY-MM-DD_your-article-title.md (I also removed the unique ID at the end, ex: .../your-article-title-1a2b).

The repo

I also created a github repo:

https://github.com/iggredible/dev-fetch-articles

Just clone the repo, install all dependencies, and create content/ directory in project root. Don't forget to give it your .env information!

git clone https://github.com/iggredible/dev-fetch-articles.git
cd dev-fetch-articles
npm install
mkdir content
mkdir .env
// create a key called DEV_KEY and give it your dev key values from dev.to site

You can find your DEV_KEY secret inside Settings -> Account on dev.to site.

Once done, just run npm run fetch-dev. You will find all your articles inside content/ folder you just created. You can use this script into your app or use this script to generate content that you can copy to your app.

Making it better

If you see how the code can be improved, feel free to submit PR on Github or drop in the comment below.

Thanks for reading and happy blogging! 📖🔥

© Copyright 2021 Igor Irianto