Creating an Arrow dataset
An exploration of the file formats that Arrow can read and write.
Posts by Category
See also:
Arrow Exploration
Arrow exploration
How to use Arrow to work with large CSV files?
A short practical guide to load a 15 GB dataset with Apache Arrow using R and Python.
Creating an Arrow dataset (part 2)
How does partitioning impact query performance?
Cukes in the news
Sea cucumber podcast at the Encyclopedia of Life
I just found this podcast about sea cucumbers on the Encyclopedia of Life (EOL) website. It provides a nice, short and fun introduction to these wonderful an...
Stichopus herrmanni releasing gametes
Have a look at this video showing Stichopus herrmanni releasing its gametes. This event was captured by Emmanuel Tardy in New Caledonia. ...
Hacking
How to work with remote Parquet files with the duckdb R package?
Learn how to work with Parquet files over HTTPS using duckdb and dplyr.
foghorn 1.3.1 released
New version of foghorn provides access to Win-builder queue
Advent of Code 2018
Solutions for the 2018 Advent of Code
A list of resources to learn more about R programming (and other things)
A non-exhaustive semi-curated list of useful books/websites for R programming, data analysis, and working at the shell.
How to setup magithub if you have GitHub 2-factor authentication enabled?
The steps involved to create the authinfo.gpg file used by magithub when you have 2FA enabled on GitHub
Advent of Code: Day 21
Solution for Day 21 of Advent of Code
Advent of Code: Day 20
Solution for Day 20 of Advent of Code
Advent of Code: Day 19
Solution for Day 19 of Advent of Code
Advent of Code: Day 18
Solution for Day 18 of Advent of Code
Advent of Code: Day 17
Solution for Day 17 of Advent of Code
Advent of Code: Day 16
Solution for Day 16 of Advent of Code
Advent of Code: Day 15
Solution for Day 15 of Advent of Code
Advent of Code: Day 14
Solution for Day 14 of Advent of Code
Advent of Code: Day 13
Solution for Day 13 of Advent of Code
Advent of Code: Day 12
Solution for Day 12 of Advent of Code
Advent of Code: Day 11
Solution for Day 11 of Advent of Code
Advent of Code: Day 10
Solution for Day 10 of Advent of Code
Advent of Code: Day 9
Solution for Day 9 of Advent of Code
Advent of Code: Day 8
Solution for Day 8 of Advent of Code
Advent of Code: Day 7
Solution for Day 7 of Advent of Code
Advent of Code: Day 6
Solution for Day 6 of Advent of Code
Advent of Code: Day 5
Solution for Day 5 of Advent of Code
Advent of Code: Day 4
Solution for Day 4 of Advent of Code
Advent of Code: Day 3
Solution for Day 3 of Advent of Code
Advent of Code: Day 2
Solution for Day 2 of Advent of Code
Advent of Code: Day 1
Solution for Advent of Code
Demonstration of tidytext using Darwin’s “On the Origin of Species”.
A short demonstration of the R package tidytext using the 1st edition of ‘the Origin of Species’.
riceware: you can now use R to generate strong (and easy to remember) passwords
Last week I read this article on how to generate strong (but easy to remember) passwords. The method coined Diceware uses a 6-faced dice to draw a series of ...
rncl: A new package to import NEXUS and Newick trees in R
rncl’ is now officially on CRAN. This package provides an interface with the NEXUS Class Library (NCL). NCL, written in C++, is included in several popular p...
How to install BEAGLE on Ubuntu 14.04 (and make it work with BEAST)
BEAGLE is a library that computes efficiently the likelihoods needed during maximum-likelihood or Bayesian phylogenetic tree estimation. It is used by MrBaye...
How to do UBSAN and Valgrind tests on a R package that includes C/C++ code before submitting to CRAN
After a trying to submit phylobase to CRAN, I learned a lot about the quality checks that go into a package before being available to the public. Beyond the ...
Offlineimap 6.4.1 and nested labels in Gmail
I just upgraded to offlineimap 6.4.1 and I was getting the following error message: “ERROR: INFINITE FOLDER CREATION DETECTED! Folder ‘.foo’ (repository ‘gma...
Evolution 3 and offlineimap
I use offlineimap to fetch my emails from my Gmail account. I use labels as folders, some are nested, and some contains a dot in their names. I have about 12...
Ubuntu color palette for OpenOffice
I like the colors chosen by Ubuntu and I tend to use them for my presentations and posters. So, I created a palette that can be imported in OpenOffice. H...
Garli 0.96b8 patched to be compiled in Ubuntu
Since Hardy, Ubuntu includes gcc 4.3 which makes the compilation of Garli 0.96 to fail. Derrick Zwickl provides a solution to fix the problem on the Garli FA...
Photo
Southern Line Islands expedition photos
Some colorful animals seen in the Southern Line Islands
Happy Halloween – Hibana gracilis
<a href= "https://lh6.ggpht.com/-T_3kkUWVHzw/Tq77B-4IJII/AAAAAAAACZ4/CIdExKGQmG8/s800/IMG_7086%252520%252528Modified%252520in%252520GIMP...
Photos: Heron & Egret
<a href= "https://lh3.ggpht.com/_DiOeIryp5ac/TPJ9Rw75iMI/AAAAAAAACLA/g_exTFWqQzU/s800/IMG_8397.jpg" > <img src= ...
Photos: Thanksgiving birding
<a href= "https://lh6.ggpht.com/_DiOeIryp5ac/TO6Qxn6l4uI/AAAAAAAACJQ/P8Y32Da4r0k/s800/IMG_8574.jpg" > <img src= ...
Photos: Millipedes
<a href= "https://lh5.ggpht.com/_DiOeIryp5ac/TO2zyXreZeI/AAAAAAAACIU/jYCq67x8ZXc/s800/IMG_8547.jpg" > <img src= ...
R in Production
Migrate from Gmail to HelpScout with R
How R allowed The Carpentries to migrate emails from Gmail to HelpScout using their web APIs
Recent articles
Revision of Phyrella and a new species: Phyrella mookiei
This post is about my latest paper, that you can download here Revision of the genus Phyrella (Holothuroidea: Dendrochirotida) with the description of a new...
The sea cucumbers from Guam
Maybe more than any other island in the tropical Pacific, Guam has been the site of many studies about its sea cucumbers. For instance, Guam is the type loca...
What do crown-of-thorns do to an ecosystem?
Crown-of-thorns (Acanthaster planci) are sea stars living on coral reefs. They can grow very large (more than 50cm/1.5ft in diameter) and they feed on corals...
Palm trees across Wallace’s line
Alfred Russel Wallace was a British naturalist who lived at the same time as Darwin. His travels to the Amazonian forest and the Malay archipelago led him to...
The éclair sea cucumber in Okinawa
A new paper with our Japanese colleagues was recently published in the Biological Magazine of Okinawa. It’s the first record of Holothuria nigralutea after i...
Spineless Science
From the field: In the sea, comets become stars.
I’m currently in Papua New Guinea, with John (the Slapcinsky kind) where we are participating to the expedition “Our planet reviewed” organized by the Muséum...
Trip to Okinawa #2 – How do you tell species apart? (what does the DNA say?)
In the previous post, I introduced you to the species complex Holothuria edulis with the three players: the pink sausage, the gray one and the éclair. If one...
Trip to Okinawa #1 – How do you tell species apart?
I just got back from a trip to Okinawa. In the next week, I will report on the reason for my visit and some of the findings. The island of Okinawa is part of...
National Geographic article about the Biocode project
National geographic just published online a series of articles and photos illustrating the Biocode project. This initiative seeks to document all the species...
Sea cucumber workshop in Guam
Part of the lab is currently in Guam because of sea cucumbers. In the context of the PEET project, we brought together 15 sea cucumber specialists and stude...
Sea cucumber workshop
I’m in Guam for the sea cucumber workshop. We are gathering experts and students to share the latest knowledge on the biology, ecology and taxonomic status o...
Invertebrates in the News #5 (follow up)
Last week, I told you that Terry Gosliner was blogging about his field trip in the Philippines where he is looking for nudibranchs (sea slugs). Today, there ...
Invertebrates in the News #6 - DNA reveals the complex life cycle of the stalked jellyfishes
If I say jellyfish, you probably picture a Frisbee-like animal with long tentacles that swims gracefully in an open ocean. </div> However, there i...
Invertebrates in the News #5
Currently, the New York Times is hosting a blog series entitled “Scientists at work, notes from the field”. Today, Terry Gosliner from the California Academy...
Invertebrates in the news #4 - Speciation in reef hermit crabs
The journal Science has a section entitled “Editor’s choice” where they feature recent papers that caught their attention. This week, they chose to highli...
Invertebrates in the news #3 - A new tree of life for the Arthropods
You may not have heard of the word “arthropod” but you are certainly familiar with at least some members of this group: the insects. While insects make arthr...
Invertebrates in the news #2 - David Liittschwager’s images in National Geographic
For this second edition of invertebrates in the news, I chose to follow up on a previous post. In early December, Seabird reported on David Liittschwager’s v...
Invertebrates in the news #1 - Bdelloid rotifers & Corynactis viridis
To keep you entertained about invertebrates when we are not in the field, I will write regularly about invertebrates that made the news recently. On the menu...
Tonna perdix eating Stichopus sp.
Not many animals are known to eat sea cucumbers. There is a good reason. Most of them have chemicals in their body, which probably doesn’t taste too good but...
Teaching
A tutorial to get started with species delineation methods using GMYC
I wrote a tutorial to get started using GMYC (the Generalized Mixed Yule Coalescent). GMYC is a popular method for species delineation based on a single locu...
Reproducible Science Workshop at iDigBio
Cross-posted from the iDigBio blog. Reproducibility is one of the corner stones of the scientific process. Readers should be able to figure out how you gen...
Tutorial
The specimen workflow: from field to database
A write-up on data collection for large biodiversity surveys
How to use the iDigBio portal to look for taxa?
An illustrated demonstration on how to get taxa from a specific geographic zone using the iDigBio portal.
Updates
What I read in 2013
Here is a list of the favorite articles I read this year. Our Silver-Coated Future, Robin Marantz Henig How nano-technology is slowly becoming ubiquitous i...
Southern Line Islands expedition photos
Some colorful animals seen in the Southern Line Islands
Molecular Evolution workshop
I’m at Woods Hole, MA for the Molecular Evolution Workshop.
Down-under
I started the year in summer as I was in the southern hemisphere. First, I participated to the 13th International Echinoderm Conference in Hobart, Tasmania. ...
Community phylogenetics seminar
This Fall, I organized with Ben Bolker a seminar on community phylogenetics. You can see the paper we read here.
Summer activities
Since I came back from my field trip in the Indian Ocean, I have been busy. First, I went to the 11th International Coral Reef Symposium (ICRS) in Fort Laude...
Trip to the Indian Ocean
Tim Werner (University of Boston) wrote a blog for our recent expedition in Nosy Bé (Madagascar). After this trip to Madagascar, Arthur Anker and I spent two...
Morphbank workshop
On May 1st and 2nd, I participated to the MorphBank usability workshop. It took place at FSU in Tallahassee. It was the occasion to interact with the develop...
My website is launched
I decided to start a website to talk about my research and about the organisms I study for my PhD: the sea cucumbers. In addition of the things usually found...