Once your VM is up and running you can test Deep using the shell. This distribution will include Stratio's customized Cassandra distribution containing our powerful open-source lucene-based secondary indexes, see Stratio documentation for further information. We also distribute the VM with several preloaded datasets in Cassandra. This VM will work on both Virtualbox and VMWare, and comes with a fully configured distribution that also includes Stratio Deep.
DOWNLOAD SPARK MAVEN DOWNLOAD
Download a pre-configured Stratio platform VM Stratio's BigData platform (SDS).Put Deep to work on a working cassandra + spark cluster.
DOWNLOAD SPARK MAVEN INSTALL
Enter deep-parent subproject and perform: mvn clean install (add -DskipTests to skip tests)
ORM API, you just have to annotate your POJOs with Deep annotations and magic will begin, you will be able to connect MongoDB with Spark using your own model entities. Support for MongoDB has been added in version 0.3.0. Spark-MongoDB connector is based on Hadoop-mongoDB.
DOWNLOAD SPARK MAVEN HOW TO
Please, refer to the deep-example project README for further information on how to setup a working environment. We encourage you to read the more comprehensive documentation hosted on the Openstratio website.ĭeep comes with an example sub project called 'deep-examples' containing a set of working examples, both in Java and Scala. Please, refer to the Deep API documentation to know more about the Cells and Cell objects. To get the value of column 'address' you can issue a c.getCellByName("address").getCellValue(). Once you get an instance 'c' of the Cells object, This interface is a little bit more cumbersome to work with (see the example below),īut has the advantage that it doesn't require the definition of additional entity classes.Įxample: you have a table called 'users' and you decide to use the 'Cells' interface. The second one is a more generic 'cell' API, that will let developerss work on RDD where a 'Cells' object is a collection of .Cell objects.Ĭolumn metadata is automatically fetched from the data store. Your domain entities must be correctly annotated using Deep annotations (take a look at deep-core example entities in package .entity). This abstraction is quite handy, it will let you work on RDD (under the hood Deep will transparently map Cassandra's columns to entity properties). We call this API the 'entity objects' API. The first one will let developers map Cassandra tables to plain old java objects (POJOs), just like if you were using any other ORM. The integration is not based on the Cassandra's Hadoop interface.ĭeep comes with an user friendly API that lets developers create Spark RDDs mapped to Cassandra column families. We actually support Apache Cassandra, MongoDB, Elastic Search, Aerospike, HDFS, S3 and any database accessible through JDBC, but in the near future we will add support for sever other datastores. tar.gz View on GitHubĭeep is a thin integration layer between Apache Spark and several NoSQL datastores. Deep-spark Connecting Apache Spark with different data stores Download.
0 kommentar(er)
