Chapter 4. Big Data – Advanced Analytics

In this chapter, we will deal with one of the biggest challenges of high-performance financial analytics and data management; that is, how to handle large datasets efficiently and flawlessly in R.

Our main objective is to give a practical introduction on how to access and manage large datasets in R. This chapter does not focus on any particular financial theorem, but it aims to give practical, hands-on examples to researchers and professionals on how to implement computationally - intensive analyses and models that leverage large datasets in the R environment.

In the first part of this chapter, we explained how to access data directly for multiple open sources. R offers various tools and options to load data into the R environment without any prior data-management requirements. This part of the chapter will guide you through practical examples on how to access data using the Quandl and qualtmod packages. The examples presented here will be a useful reference for the other chapters of this book. In the second part of this chapter, we will highlight the limitation of R to handle big data and show practical examples on how to load a large amount of data in R with the help of big memory and ff packages. We will also show how to perform essential statistical analyses, such as K-mean clustering and linear regression, using large datasets.