How to import Firefox bookmarks to Nextcloud application

I am currently playing with floccus to store bookmarks using Nextcloud, so I have created simple shell scripts to import Nextcloud bookmarks from Firefox using code from my earlier solution to open Firefox bookmarks from the OpenBox menu.

Nextcloud bookmarks and floccus - Google Chrome using data exported from Firefox — Nextcloud bookmarks and floccus – Google Chrome using data exported from Firefox

Prerequisites

Install curl utility to perform API calls.

$ sudo apt-get install curl

Install jq utility to parse JSON files.

$ sudo apt-get install jq

Install sqlite3 utility to access the SQLite database.

$ sudo apt-get install sqlite3

Create an application password

Create a Nextcloud application password, do not use your regular credentials.

Bookmarks

I will use the following bookmarks in this article. These bookmarks are included in the source code.

Mozilla Firefox>Pomoc (https://support.mozilla.org/pl/products/firefox)
Mozilla Firefox>Dostosuj Firefoksa (https://www.mozilla.org/pl/firefox/customize/)
Mozilla Firefox>Dołącz do nas (https://www.mozilla.org/pl/contribute/)
Mozilla Firefox>O Mozilli (https://www.mozilla.org/pl/about/)
Ubuntu and Free Software links>Ubuntu (http://www.ubuntu.com/)
Ubuntu and Free Software links>Ubuntu Wiki (community-edited website) (http://wiki.ubuntu.com/)
Ubuntu and Free Software links>Make a Support Request to the Ubuntu Community (https://answers.launchpad.net/ubuntu/+addquestion)
Ubuntu and Free Software links>Debian (Ubuntu is based on Debian) (http://www.debian.org/)
Apps>The most popular self-hosted file share and collaboration platform (https://nextcloud.com/)
Apps>Open Source Password Management Solutions | Bitwarden (https://bitwarden.com/)
News>Welcome to LWN.net [LWN.net] (https://lwn.net/)
News>Services>NewsBlur (https://newsblur.com/)
Personal>Services>sleeplessbeastie's notes (https://sleeplessbeastie.eu/)
Debian>Debian -- The Universal Operating System (https://www.debian.org/)
Debian>Planet Debian (https://planet.debian.org/)

Display and export bookmarks

Create nextcloud_bookmarks_print.sh shell script.

Shell script

#!/bin/sh
# Export Firefox bookmarks 

# path to the sqlite3 binary
sqlite_path=$(which sqlite3)

# sqlite3 parameters (define separator character)
sqlite_params="-separator ^"

# path to the places.sqlite database
bookmarks_database=$(ls ~/.mozilla/firefox/*.default/places.sqlite)

# SQL query 
sql_query="select p.title, p.url from moz_places as p where p.hidden=0 order by last_visit_date desc limit 10"

# root element
root_element_query="select id from moz_bookmarks where rtrim(guid,'_')='menu'"
root_element="$($sqlite_path $sqlite_params "$bookmarks_database" "$root_element_query")"

# process bookmarks
process_bookmarks(){
  # SQL query - folders
  folder_id=$1
  folder_path=""
  while [ "$folder_id" != "2" ]; do
    sql_folder_query="select parent,title from moz_bookmarks where id=$folder_id and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"

    sql_folder_result=$($sqlite_path $sqlite_params "$bookmarks_database" "$sql_folder_query" )
    folder_id=$(echo $sql_folder_result | awk -F^ '{print $1}')
    folder_title=$(echo $sql_folder_result | awk -F^ '{print $2}')

    # special case for empty title
    if [ -z "$folder_path" ]; then
      folder_path="floccus:>$folder_title"
    else
      folder_path="${folder_path}>${folder_title}"
    fi
  done

  # escape special characters
  folder_path=$(echo $folder_path | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")

  # SQL query - bookmarks
  sql_bookmarks_query="select b.title, p.url from moz_bookmarks as b left outer join moz_places as p on b.fk=p.id where b.type = 1 and p.hidden=0 and b.title not null and parent=$1"
  $sqlite_path $sqlite_params "$bookmarks_database" "$sql_bookmarks_query" | while IFS=^ read title url; do
    # special case for empty title
    if [ -z "$title" ]; then
      title=$url
    fi

    # escape special characters
    title=$(echo $title | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")
    url=$(echo $url | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")

    e display url, title, path
    echo "<DT><A HREF=\"${url}\" TAGS=\"${folder_path}\">${title}</A>"
  done
}

# process the folders function
process_folders(){
  # execute only when there is an exactly one parameter
  if [ "$#" = 1 ]; then
    # SQL query - folders
    sql_folder_query="select id from moz_bookmarks where parent=$1 and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"

    # process folders
    $sqlite_path $sqlite_params "$bookmarks_database" "$sql_folder_query" | while IFS=^ read id; do
      # process folders inside
      process_folders $id
      
      # process bookmarks in current folder
      process_bookmarks $id 
    done
  fi
}


# header
echo "<!DOCTYPE NETSCAPE-Bookmark-file-1>"
echo "<META HTTP-EQUIV=\"Content-Type\" CONTENT=\"text/html; charset=UTF-8\">"
echo "<!-- This is an automatically generated file."
echo "It will be read and overwritten."
echo "Do Not Edit! -->"
echo "<TITLE>Bookmarks</TITLE>"
echo "<H1>Bookmarks</H1>"
echo "<DL><p>"

# process folders 
process_folders "$root_element"

# process bookmarks for root element
process_bookmarks "$root_element"

Usage

Export Firefox bookmarks to specified filename using floccus tags to preserve path.

$ nextcloud_bookmarks_print.sh | tee export.html
<!DOCTYPE NETSCAPE-Bookmark-file-1>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">
<!-- This is an automatically generated file.
It will be read and overwritten.
Do Not Edit! -->
<TITLE>Bookmarks</TITLE>
<H1>Bookmarks</H1>
<DL><p>
<DT><A HREF="https://support.mozilla.org/pl/products/firefox" TAGS="floccus:>Mozilla Firefox">Pomoc</A>
<DT><A HREF="https://www.mozilla.org/pl/firefox/customize/" TAGS="floccus:>Mozilla Firefox">Dostosuj Firefoksa</A>
<DT><A HREF="https://www.mozilla.org/pl/contribute/" TAGS="floccus:>Mozilla Firefox">Dołącz do nas</A>
<DT><A HREF="https://www.mozilla.org/pl/about/" TAGS="floccus:>Mozilla Firefox">O Mozilli</A>
<DT><A HREF="http://www.ubuntu.com/" TAGS="floccus:>Ubuntu and Free Software links">Ubuntu</A>
<DT><A HREF="http://wiki.ubuntu.com/" TAGS="floccus:>Ubuntu and Free Software links">Ubuntu Wiki (community-edited website)</A>
<DT><A HREF="https://answers.launchpad.net/ubuntu/+addquestion" TAGS="floccus:>Ubuntu and Free Software links">Make a Support Request to the Ubuntu Community</A>
<DT><A HREF="http://www.debian.org/" TAGS="floccus:>Ubuntu and Free Software links">Debian (Ubuntu is based on Debian)</A>
<DT><A HREF="https://nextcloud.com/" TAGS="floccus:>Apps">The most popular self-hosted file share and collaboration platform</A>
<DT><A HREF="https://bitwarden.com/" TAGS="floccus:>Apps">Open Source Password Management Solutions | Bitwarden</A>
<DT><A HREF="https://lwn.net/" TAGS="floccus:>News">Welcome to LWN.net [LWN.net]</A>
<DT><A HREF="https://newsblur.com/" TAGS="floccus:>News>Services">NewsBlur</A>
<DT><A HREF="https://sleeplessbeastie.eu/" TAGS="floccus:>Personal>Services">sleeplessbeastie's notes</A>
<DT><A HREF="https://www.debian.org/" TAGS="floccus:>Debian">Debian -- The Universal Operating System</A>
<DT><A HREF="https://planet.debian.org/" TAGS="floccus:>Debian">Planet Debian</A>

You can import this file to Nextcloud Bookmarks using the web-interface or restore the Nextcloud bookmarks shell script.

Use Nextcloud API to import bookmarks from Firefox

This automated solution will use API to import Firefox bookmarks directly to the Nextcloud bookmarks application. Create firefox_bookmarks_add.sh shell script.

Shell script

#!/bin/sh
# Import Firefox bookmarks to Nexcloud
# https://sleeplessbeastie.eu/

# path to the sqlite3 binary
sqlite_path=$(which sqlite3)

# sqlite3 parameters (define separator character)
sqlite_params="-separator ^"

# path to the places.sqlite database
bookmarks_database=$(ls ~/.mozilla/firefox/*.default/places.sqlite)

# SQL query 
sql_query="select p.title, p.url from moz_places as p where p.hidden=0 order by last_visit_date desc limit 10"

# root element
root_element_query="select id from moz_bookmarks where rtrim(guid,'_')='menu'"
root_element="$($sqlite_path $sqlite_params "$bookmarks_database" "$root_element_query")"

# escape html
escape_html() {
  echo $* | perl -n -mHTML::Entities -e "print HTML::Entities::encode_entities_numeric(\$_,'<>&\"\'[](){}#@|%+')"
}

# process bookmarks
process_bookmarks(){
  # create folder path
  folder_id=$1
  folder_path=""
  while [ "$folder_id" != "2" ]; do
    sql_folder_query="select parent,title from moz_bookmarks where id=$folder_id and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"

    sql_folder_result=$($sqlite_path $sqlite_params "$bookmarks_database" "$sql_folder_query" )
    folder_id=$(echo $sql_folder_result | awk -F^ '{print $1}')
    folder_title=$(echo $sql_folder_result | awk -F^ '{print $2}')

    # special case for empty title
    if [ -z "$folder_path" ]; then
      folder_path="floccus:>$folder_title"
    else
      folder_path="${folder_path}>${folder_title}"
    fi
  done

  # process bookmarks
  sql_bookmarks_query="select b.title, p.url from moz_bookmarks as b left outer join moz_places as p on b.fk=p.id where b.type = 1 and p.hidden=0 and b.title not null and parent=$1"
  $sqlite_path $sqlite_params "$bookmarks_database" "$sql_bookmarks_query" | while IFS=^ read ff_title ff_url; do
    # check if url is already stored before adding it
    found_bookmark="0"
    continue_pagination="1"
    page=0
    while [ "${continue_pagination}" -eq "1" ]; do
      urls=$(curl --silent -X GET --user "${param_username}:${param_password}" \
                                  --header "Accept: application/json" \
                  "${param_nextcloud_address}/index.php/apps/bookmarks/public/rest/v2/bookmark?page=${page}&search\[\]=$(escape_html $ff_url)" | \
             jq -r '.data[].url')
      if [ -z "${urls}" ]; then
        continue_pagination="0"
      else
        for url in $urls; do
          if [ "$ff_url" == "$url" ]; then
            found_bookmark="1"
            break 
          fi
        done
      fi

      if [ "${found_bookmark}" -eq "0" ]; then
        status=$(curl --silent -X POST --user "${param_username}:${param_password}" \
                                       --data-urlencode "url=$ff_url" \
                                       --data-urlencode "title=$ff_title" \
                                       --data-urlencode "item[tags][]=$folder_path" \
                      "${param_nextcloud_address}/index.php/apps/bookmarks/public/rest/v2/bookmark" | \
                 jq -r 'select(.status != "success") | .status')
        if [ -n "${status}" ]; then
          echo "Skipped Nextcloud bookmark url \"${ff_url}\" with title \"${ff_title}\" and tag \"${folder_path}\"."
        else
          echo "Added Nextcloud bookmark url \"${ff_url}\" with title \"${ff_title}\" and tag \"${folder_path}\"."
        fi
        continue_pagination="0"
        page="0"
      else
        page=$(expr $page + 1)
      fi
    done
  done
}

# process the folders function
process_folders(){
  # execute only when there is an exactly one parameter
  if [ "$#" = 1 ]; then
    # SQL query - folders
    sql_folder_query="select id from moz_bookmarks where parent=$1 and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"

    # process folders
    $sqlite_path $sqlite_params "$bookmarks_database" "$sql_folder_query" | while IFS=^ read id; do
      # process folders inside
      process_folders $id
      
      # process bookmarks in current folder
      process_bookmarks $id 
    done
  fi
}

# usage info
usage(){
  echo "Usage:"
  echo "  $0 -r nextcloud_url -u username -p passsword"
  echo ""
  echo "Parameters:"
  echo "  -r nextcloud_url   : set Nextcloud URL (required)"
  echo "  -u username        : set username (required)"
  echo "  -p password        : set password (required)"
  echo ""
}

# parse parameters
while getopts "r:u:p:" option; do
  case $option in
    "r")
      param_nextcloud_address="${OPTARG}"
      param_nextcloud_address_defined=true
      ;;
    "u")
      param_username="${OPTARG}"
      param_username_defined=true
      ;;
    "p")
      param_password="${OPTARG}"
      param_password_defined=true
      ;;
    \?|:|*)
      usage
      exit
      ;;
  esac
done

if [ "${param_nextcloud_address_defined}" = true ] && \
   [ "${param_username_defined}"          = true ] && \
   [ "${param_password_defined}"          = true ]; then

  # process folders 
  process_folders "$root_element"

  # process bookmarks for root element
  process_bookmarks "$root_element"
else
  usage
fi

Usage

Display usage information.

$ firefox_bookmarks_add.sh 
Usage:
  firefox_bookmarks_add.sh -r nextcloud_url -u username -p passsword

Parameters:
  -r nextcloud_url   : set Nextcloud URL (required)
  -u username        : set username (required)
  -p password        : set password (required)

Import Firefox bookmarks.

$ firefox_bookmarks_add.sh -r https://cloud.example.org/ -u milosz -p Mjdu3-kDnru-4UksA-fYs0w
Added Nextcloud bookmark url "https://support.mozilla.org/pl/products/firefox" with title "Pomoc" and tag "floccus:>Mozilla Firefox".
Added Nextcloud bookmark url "https://www.mozilla.org/pl/firefox/customize/" with title "Dostosuj Firefoksa" and tag "floccus:>Mozilla Firefox".
Added Nextcloud bookmark url "https://www.mozilla.org/pl/contribute/" with title "Dołącz do nas" and tag "floccus:>Mozilla Firefox".
Added Nextcloud bookmark url "https://www.mozilla.org/pl/about/" with title "O Mozilli" and tag "floccus:>Mozilla Firefox".
Added Nextcloud bookmark url "http://www.ubuntu.com/" with title "Ubuntu" and tag "floccus:>Ubuntu and Free Software links".
Added Nextcloud bookmark url "http://wiki.ubuntu.com/" with title "Ubuntu Wiki (community-edited website)" and tag "floccus:>Ubuntu and Free Software links".
Added Nextcloud bookmark url "https://answers.launchpad.net/ubuntu/+addquestion" with title "Make a Support Request to the Ubuntu Community" and tag "floccus:>Ubuntu and Free Software links".
Added Nextcloud bookmark url "http://www.debian.org/" with title "Debian (Ubuntu is based on Debian)" and tag "floccus:>Ubuntu and Free Software links".
Added Nextcloud bookmark url "https://nextcloud.com/" with title "The most popular self-hosted file share and collaboration platform" and tag "floccus:>Apps".
Added Nextcloud bookmark url "https://bitwarden.com/" with title "Open Source Password Management Solutions | Bitwarden" and tag "floccus:>Apps".
Added Nextcloud bookmark url "https://lwn.net/" with title "Welcome to LWN.net [LWN.net]" and tag "floccus:>News".
Added Nextcloud bookmark url "https://newsblur.com/" with title "NewsBlur" and tag "floccus:>News>Services".
Added Nextcloud bookmark url "https://sleeplessbeastie.eu/" with title "sleeplessbeastie's notes" and tag "floccus:>Personal>Services".
Added Nextcloud bookmark url "https://www.debian.org/" with title "Debian -- The Universal Operating System" and tag "floccus:>Debian".
Added Nextcloud bookmark url "https://planet.debian.org/" with title "Planet Debian" and tag "floccus:>Debian".

Additional notes

Download source code, Firefox bookmarks exported in json and html format are included too.

Remember to backup Nextcloud bookmarks.

Prerequisites#

Create an application password#

Bookmarks#

Display and export bookmarks#

Shell script#

Usage#

Use Nextcloud API to import bookmarks from Firefox#

Shell script#

Usage#

Additional notes#

Prerequisites

Create an application password

Bookmarks

Display and export bookmarks

Shell script

Usage

Use Nextcloud API to import bookmarks from Firefox

Shell script

Usage

Additional notes