Importing formatted data files using the functions in the foreign package

The foreign package contains functions that will allow you to import data files from some of
the most commonly used statistical software packages such as SAS, Stata and SPSS.
To download the foreign package from the CRAN website from within R, click on
“Packages” and then “Install package(s) from CRAN”.  You will then need to
load the package, and you can use the help function.

library(foreign)
help(package=foreign)

The package contains the following functions:

data.restore   Read an S3 Binary File
lookup.xport   Lookup Information on a SAS XPORT Format
               Library
read.dbf       Read a DBF File
read.dta       Read Stata binary files
read.epiinfo   Read Epi Info data files
read.mtp       Read a Minitab Portable Worksheet
read.octave    Read Octave Text Data Files
read.spss      Read an SPSS data file
read.ssd       Obtain a Data Frame from a SAS Permanent
               Dataset, via read.xport
read.systat    Obtain a Data Frame from a Systat File
read.xport     Read a SAS XPORT Format Library
write.dbf      Write a DBF File
write.dta      Write Files in Stata Binary Format
write.foreign  Write text files and code to read them

To download the package:
install.packages("foreign")

To view the functions in the package:
library(help=foreign)

To view the help file for a specific function, for example the function read.dta:
?read.dta

Note that this is an outdated package and will not work for Stata 12 or higher data files. There is a package called “readstata13” available to read Stata 13 files.

Here are examples of importing a Stata data file called
test.dta.

test.stata <- read.dta("https://stats.idre.ucla.edu/stat/data/test.dta")
print(test.stata)

   make   model mpg weight price
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

Importing free formatted (delimited) data files using the read.table function

The read.table function is very useful when reading in ASCII files that contain rectangular data.
When the file contains the variable names in the first line of data the option header should be set to
TRUE. The default delimiter is blank space, other delimiters must be specified by using the sep option and setting it equal to the delimiter in quotes (i.e., sep=”;” for the semicolon delimited data file).

Here are some examples of data with different types of delimiters. We will start by looking at
a typical bread and butter type of data file namely a space delimited ASCII file
called test.txt.

test.txt <- read.table("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/test.txt", header=T)
print(test.txt)

   make   model mpg weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

Another very common type of file is the comma delimited file. The file
test-1.csv has been saved out of Excel as a comma delimited file.
This file can be read in by the read.table function by using the sep option,
but it can also be read in by the read.csv function which was written specifically for
comma delimited files.

test.csv <- read.csv("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/test-1.csv", header=T)
print(test.csv)

   make   model mpg weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

test.csv1 <- read.table("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/test-1.csv", header=T, sep=",")
print(test.csv1)

   make   model mpg weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

It is, of course, also possible to use the read.table function for
reading in files with other delimiters. In the data called
testsemicolon.txt has semicolon delimiters and
the dataset test called testz.txt uses the letter z as a
delimiter, both of which are acceptable delimiters in R.

test.semi <- read.table("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/testsemicolon.txt", header=T, sep=";")
print(test.semi)

   make   model mpg weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

test.z <- read.table("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/testz.txt", header=T, sep="z")
print(test.z)

   make   model mpg weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

Importing data files using the scan function

The scan function is an extremely flexible tool for importing data. It can be used
to read in almost any type of data, numeric, character or complex and it can be
used for fixed or free formatted files. Moreover, by using the scan function it is
possible to input data directly from the console. The scan function reads the fields of
data in the file as specified by the what option with the default being numeric. If
the what option is specified to be what=character() or what=” ” then all the
fields will be read as strings. If the data is a mix of numeric, string or complex data then
a list can be used in the what option. The default separator for the scan function
is any white space (single space, tab, or new line). However, unlike the read.table
function which returns a data frame, the scan function returns a list or
a vector.  This makes the scan function less useful for inputting
“rectangular” data such as the car data set that was seen in the
previous examples.

In the following examples we input first numeric data and then string data directly from the console; then
we input the text file, scan.txt, where the first variable is a string variable and the
second variable is numeric.

#inputting data directly from the console
x <- scan()
1: 3 5 6 9
5: 2 5 6 
8: 
Read 7 items
x
[1] 3 5 6 9 2 5 6

# inputting string data directly from the console
name.x <- scan(, what="")
1: bobby
2: kate dave
4: mia
5: 
Read 4 items
name.x
[1] "bobby" "kate"  "dave"  "mia" 

# inputting a text file and outputting a list
x <- scan("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/scan.txt", what=list(age=0, name=""))
Read 4 records

x
$age
[1] 12 24 35 20

$name
[1] "bobby"   "kate"    "david"   "michael"

# using the same text file and saving only the names as a vector
x <- scan("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/scan.txt", what=list(NULL, name=character()))
Read 4 records

x <- x[sapply(x, length) > 0]x
$name
[1] "bobby"   "kate"    "david"   "michael"

is.vector(x)
[1] TRUE

Importing Fixed Format Files Using the read.fwf Function

For fixed format files the variables names are often in a separate file from the data.
In this example the variable names are in a file called names.txt and the
data are in a file called testfixed.txt.  This is especially
convenient when the fixed format file is very large and has many variables; then it becomes
rather impractical to type in all the variable names.  In this
situation the width option is used to specify the width of each
variable and the col.name option specifies the file containing the
variable names.  So, first we read in the file for the names using the
scan function.  We specify that file contains character values by
setting the what option to equal character().  By using the
col.names option in the read.fwf function names will supply
the variables names.

names <- scan("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/names.txt", what=character() )
Read 5 items
print(names)

[1] "model"  "make"   "mph"    "weight" "price" 

test.fixed <- read.fwf("https://stats.idre.ucla.edu/wp-content/uploads/2016/02/testfixed.txt", col.names=names, width = c(5, 7, 2, 4, 4))
print(test.fixed)

  model    make mph weight price 
1   AMC Concord  22   2930  4099
2   AMC   Pacer  17   3350  4749
3   AMC  Spirit  22   2640  3799
4 Buick Century  20   3250  4816
5 Buick Electra  15   4080  7827

Note that if there is a warning about an incomplete final line, it means you should add a carriage return (press Enter) after the last line so that the function will know when to stop reading the file.

Exporting files using the write.table function

The write.table function outputs data files. The first argument specifies which
data frame in R is to be exported. The next argument specifies the file to be created.
The default separator is a blank space but any separator can be specified in the sep option.
The default value for both the row.names and col.names options is TRUE. In the example we specify
that we do not wish to include row names. The default setting for the quote option is to include
quotes around all the character values, i.e. around values in string variables and around the
column names. As we have shown in the example it is very common not to want the quotes when
creating a text file.

# using the test.csv data frame to write a text file with no row names 
# and without quotes around the character values (both column names and string variables)

write.table(test.csv, "c:/temp/test1.txt", row.names=F, quote=F)