java
Importing Data from different sources - SOA approach description
To make full use of the basic principles of the service oriented architecture it is useful to define a public/generally accepted model of the data object to be stored. This model has to encompass all (or at least most) of the possible data such an object could have and is later handled in the lowest level of the architecture, the data layer. In this layer the data objects can be handled by a database, so let’s assume we use JAVA as the basic language, the data objects could then be stored in an HSQLDB, accessible by a basic CRUD-interface.
Above, in the service layer a general import/export function must be defined, and while this general function should not be bound to one format it will have to work through the data object already described. This is, most often than not, the hardest part as these functions must cover and handle model specific data in an open way as to not limit the presentation of the data in the next layer and to still allow the CRUD to handle the data object without loss of information.
It was already mentioned that the next layer will be the presentation layer. This is the first layer where actual information on the various formats is used. In object oriented programming languages it is quite easy to realize this (abstract classes in Java for example), once the underlying layers are working properly. Here is also the only location in the program where there is a true difference in the representation of the data. While it was quite abstract and handled only to store and retrieve in the layers below, here it can be manipulated to fit certain formats or vies-versa so that certain formats fit the underlying data object.
By this approach it becomes possible to expand (and reduce) the formats a service is able to manage without changing the underlying service. This has a positive effect on the work time as well as consistency throughout such projects.
Twitter your project status with Maven.
Create a simple hello world Java Maven Project (this is already available as an maven-archetype) and configure it in a way so that every time you make a installation (via the cli command: mvn clean install) a new status update with the message "New Project Built Available" shows up on your twitter account.
Taggings:
OutOfMemoryException when using WEKA 3-6 under Windows 7
Firstly the Weka-User has to extend the memory available for the Java virtual machine by setting appropriate options. With Sun's JDK the Weka-User can set the parameter "-Xmx1024m" to set the maximum Java heap size to 1024MB.
Under Windows 7 The Weka-User can set this parameter under Control Panel -> Java -> Java -> View -> Runtime Parameters.
Secondly the Weka-User has to change the "maxheap" parameter in the RunWeka.ini File. This file is contained in the root directory of your Weka installation. By setting "maxheap=1024m" the maximum heap size is set to 1024MB.
Taggings:
OutOfMemoryException when using WEKA 3-6 under Windows 7
Weka is a collection of machine learning algorithms for data mining tasks. This software can be used by students who participate in the class "Machine Learning" at TU Wien.
When the Weka-User executes memory-intesitive tasks (e.g. visualizes results of a machine learning algorithm), Weka is very likely to throw an OutOfMemoryException. As a result to the exception the software crashes. This execption is absolutely repeatable. The Weka-User will get the exception at every time he executes the sequence of operations which had previously lead to the exeception. That is why there is no way to workaround this error without having to disclaim using Weka. The OutOfMemoryException is not only limited to Weka 3-6 on Windows 7.
Make multiple requests by using asynchronous threads
It is possible to make multiple requests by using multithreading.
For managing concurrency you can use ExecutorService
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/Execut...
Example:
ExecutorService executor = Executors.newFixedThreadPool(size);
Future response1 = executor.submit(new GetComments(offset));
Future response2 = executor.submit(new GetComments(offset));
.....
class GetComments implements Callable {
private String url;
public ParralelCommentsRequest(int offset) {
this.url = ....
}
@Override
public InputStream call() throws Exception {
return new URL(url).openStream();
}
}
The method submit extends base method Executor.execute(java.lang.Runnable) by creating and returning a Future that can be used to cancel execution and/or wait for completion.
The class GetComments implements Callable which is similar to Runnable, but it can return a result.
Taggings:
NY Times Community API only returns 25 comments per request
The Community API from The New York Times allows to retrieve user comments from articles. I want to collect all comments from a given date and analyze the content, by looking at the most common words and display them in a tag cloud.
Now the API allows me to search comments by date, but the problem is that it only returns 25 comments. The problem is that on some days roughly 5000 comments are submitted and the API doesn't give me the option to fetch them all at once. The only way to get more comments is by using the parameter "offset". This parameter allows me to set the starting point of the result set and is a multiple of 25, e.g. offset=25 displays me comments 26-50.
Fetching all comments in this way needs too long, because each request needs ~1.5 seconds.
I am using a web interface to present the results and collect the data on a server using java.
introduction to OWL API
-What is OWL API: Java API to work in OWL 2.0 DL
-Why we need OWL API:
+The use of a higher level data model can help to
+ insulate us from the vagaries of concrete syntax.
+ make it clear what is happening in terms of functionality.
+ increase the likelyhood of interoperating applications.
-OWL Abstract Syntax
+Provides a definition of the language in terms of the
constructs and assertions allowed.
+Semantics are then defined in terms of this abstract
syntax.
+Our OWL API data model is based largely on this abstract
syntax presentation.
+Conceptually cleaner.
+Syntax doesn’t get in the way
-example:
The following examples serve to illustrate various aspects
of the API and how we might use it.
1. Producing a basic hierarchy
2. Adding Closure Axioms
3. Rendering Alternative Concrete Syntaxes
4. Black Box Debugging
We won’t go into all the details of how these are done
Source for the examples will be available on line (OWPAPI.org)
programming with OWL API
-What is OWLAPI
-Why we need OWL API
-How can we use OWP API
-Example
Taggings:
Use Java TCP Socket and Java Datagrams
The clients do not have direct access to the file servers - they only communicate with a single proxy.
This proxy handles client authentification, and client requests. When a client requests a file to be downloaded,
the proxy checks, if this is possible and chooses a file server with respect to optimal load balancing from where the requested file is transferred to the proxy. From the proxy the file is then transferred to the client.
Communication necessary for user authentification and file transfer is implemented via JAVA TCP sockets
(see http://download.oracle.com/javase/tutorial/networking/sockets/index.html ) .
Furthermore each file server shall send messages to the proxy periodically, to indicate that it is still online.
This is taken into account, when the proxy selects a server in order to download files.
For this communication the UDP protocol us used (see http://download.oracle.com/javase/tutorial/networking/datagrams/index.html )
Using DOM parser
import java.io.File;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
public class XMLReader {
public static void main(String argv[]) {
try {
File file = new File("c:\\MyXMLFile.xml");
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(file);
doc.getDocumentElement().normalize();
System.out.println("Root element " + doc.getDocumentElement().getNodeName());
NodeList nodeLst = doc.getElementsByTagName("employee");
System.out.println("Information of all employees");
for (int s = 0; s < nodeLst.getLength(); s++) {
Node fstNode = nodeLst.item(s);
if (fstNode.getNodeType() == Node.ELEMENT_NODE) {
Element fstElmnt = (Element) fstNode;
NodeList fstNmElmntLst = fstElmnt.getElementsByTagName("firstname");
Element fstNmElmnt = (Element) fstNmElmntLst.item(0);
NodeList fstNm = fstNmElmnt.getChildNodes();
System.out.println("First Name : " + ((Node) fstNm.item(0)).getNodeValue());
NodeList lstNmElmntLst = fstElmnt.getElementsByTagName("lastname");
Element lstNmElmnt = (Element) lstNmElmntLst.item(0);
NodeList lstNm = lstNmElmnt.getChildNodes();
System.out.println("Last Name : " + ((Node) lstNm.item(0)).getNodeValue());
}
}
} catch (Exception e) {
e.printStackTrace();
}
Taggings:
Pages
