prajna.data.html

Class TemplatedHtmlDocData

java.lang.Object

prajna.data.DocData

prajna.data.html.TemplatedHtmlDocData

public class TemplatedHtmlDocData
extends DocData

DocData class which supports HTML and XHTML documents. This class can load the content of the web page when the loadContent() method is called.

Author:

Edward Swing

Constructor Summary

Constructors

Constructor and Description
TemplatedHtmlDocData(java.lang.String urlString) Create a new HtmlDocData object from the provided url.
TemplatedHtmlDocData(java.net.URL url) Create a new HtmlDocData object from the provided url.

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	getBody() Gets the body.
protected java.lang.String	getBodyClass() Get the body class for this HtmlDocData
protected java.lang.String	getBodyTag() Get the body tag for this HtmlDocData
java.util.HashSet<java.net.URL>	getLinks() Get all the links contained in this HTML document
java.util.List<java.lang.String>	getSections() Return the sections of the document in order.
java.lang.String	getSectionTag() Get the section tag
java.util.Set<java.text.SimpleDateFormat>	getTimeStampFormats() Get the timestamp format
java.lang.String	getTimeStampTag() Get the timestamp tag
protected java.lang.String	getTitleTag() Get the title tag for this HtmlDocData
java.net.URL	getURL() Get the URL for this particular HtmlDocData object.
boolean	loadContent() Load the content.
void	setBodyTag(java.lang.String tag) Set the body tag.
void	setBodyTag(java.lang.String tag, java.lang.String tagClass) Set the body tag and class.
void	setSectionTag(java.lang.String tag) Set the section tag.
void	setTimeStampTag(java.lang.String metaTag, java.lang.Iterable<java.text.SimpleDateFormat> formats) Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag.
void	setTimeStampTag(java.lang.String metaTag, java.lang.String format) Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag.
void	setTimeStampTag(java.lang.String metaTag, java.lang.String[] formats) Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag.
void	setTitleTag(java.lang.String tag) Set the title tag.
TemplatedHtmlDocData	templateDocData(java.lang.String urlString) Create a new TemplatedHtmlDocData object from the provided url, using this TemplatedHtmlDocData object as a template.
TemplatedHtmlDocData	templateDocData(java.net.URL url) Create a new HtmlDocData object from the provided url, using this HtmlDocData object as a template.

Methods inherited from class prajna.data.DocData

addKey, addKey, addKeyMap, calculateSimilarity, clearKeys, equals, getKeyMap, getKeys, getLink, getName, getSummary, getTimeStamp, getValueForKey, hashCode, matchesKey, setBody, setLink, setName, setSummary, setTimeStamp, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

TemplatedHtmlDocData

public TemplatedHtmlDocData(java.lang.String urlString)
                     throws java.net.MalformedURLException

Create a new HtmlDocData object from the provided url.

Parameters:

urlString - the URL string to the document

Throws:

java.net.MalformedURLException - if the URL is not a valid URL

TemplatedHtmlDocData

public TemplatedHtmlDocData(java.net.URL url)

Create a new HtmlDocData object from the provided url.

Parameters:

url - the URL for the document

Method Detail

getBody

public java.lang.String getBody()

Gets the body. This is the content of the document.

Overrides:

getBody in class DocData

Returns:

the body

getBodyClass

protected java.lang.String getBodyClass()

Get the body class for this HtmlDocData

Returns:

the body class

getBodyTag

protected java.lang.String getBodyTag()

Get the body tag for this HtmlDocData

Returns:

the body tag

getLinks

public java.util.HashSet<java.net.URL> getLinks()

Get all the links contained in this HTML document

Returns:

the set of links in this document

getSections

public java.util.List<java.lang.String> getSections()

Return the sections of the document in order. For this simple implementation, this method returns a list containing the body as the single element

Overrides:

getSections in class DocData

Returns:

a list of document sections

getSectionTag

public java.lang.String getSectionTag()

Get the section tag

Returns:

the section tag

getTimeStampFormats

public java.util.Set<java.text.SimpleDateFormat> getTimeStampFormats()

Get the timestamp format

Returns:

the timestamp format

getTimeStampTag

public java.lang.String getTimeStampTag()

Get the timestamp tag

Returns:

the timestamp tag

getTitleTag

protected java.lang.String getTitleTag()

Get the title tag for this HtmlDocData

Returns:

the title tag

getURL

public java.net.URL getURL()

Get the URL for this particular HtmlDocData object. The URL indicates the URL reference to the document

Returns:

the URL for this DocData

loadContent

public boolean loadContent()

Load the content. This method loads the content of the document, the title, and keywords from the MTML content. The body of the document can be retrieved using the getBody() method. The title can be retrieved using getName(), and the keywords can be retrieved using getKeywords().

Overrides:

loadContent in class DocData

Returns:

true if the content can be loaded without error, false otherwise.

setBodyTag

public void setBodyTag(java.lang.String tag)

Set the body tag. The body tag is used to determine what HTML tag is used to define the body content. By default, this is set to "p".

Parameters:

tag - the tag defining the content of the document

setBodyTag

public void setBodyTag(java.lang.String tag,
              java.lang.String tagClass)

Set the body tag and class. The body tag is used to determine what HTML tag is used to define the body content. By default, this is set to "p". If the tagClass is set, the body content must match both the tag and class name to be valid content. This method is particularly useful when loading content delineated by DIV tags.

Parameters:

tag - the tag defining the content of the document

tagClass - the class name for the content

setSectionTag

public void setSectionTag(java.lang.String tag)

Set the section tag. The section tag indicates where sections, or paragraphs, stop and start. By default, this is set to "p".

Parameters:

tag - the section tag.

setTimeStampTag

public void setTimeStampTag(java.lang.String metaTag,
                   java.lang.Iterable<java.text.SimpleDateFormat> formats)

Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag. During parsing, if the tag corresponds to the NAME field of a meta tag in the HTML document, the content will be parsed using one the date format specified. The parsing will use the first format that is able to parse the string.

Parameters:

metaTag - The string corresponding to the META tag name.

formats - a set of date format strings

setTimeStampTag

public void setTimeStampTag(java.lang.String metaTag,
                   java.lang.String format)

Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag. During parsing, if the tag corresponds to the NAME field of a meta tag in the HTML document, the content will be parsed using the date format specified.

Parameters:

metaTag - The string corresponding to the META tag name.

format - a date format string indicating the time format used

setTimeStampTag

public void setTimeStampTag(java.lang.String metaTag,
                   java.lang.String[] formats)

Set the tag used to identify the time stamp in an HTML document, and a set of formats used to parse the tag. During parsing, if the tag corresponds to the NAME field of a meta tag in the HTML document, the content will be parsed using one the date format specified. The parsing will use the first format that is able to parse the string.

Parameters:

metaTag - The string corresponding to the META tag name.

formats - a set of date format strings

setTitleTag

public void setTitleTag(java.lang.String tag)

Set the title tag. Like the body tag, this is used to delineate where the title of the document is. By default the title tag is "title".

Parameters:

tag - the title tag.

templateDocData

public TemplatedHtmlDocData templateDocData(java.lang.String urlString)
                                     throws java.net.MalformedURLException

Create a new TemplatedHtmlDocData object from the provided url, using this TemplatedHtmlDocData object as a template. The new TemplatedHtmlDocData object copies the tags, formats and classes from this template. This method simply invokes templateDocData(new URL(urlString))

Parameters:

urlString - the URL string to the document

Returns:

a new HtmlDocData object that uses this object as a template

Throws:

java.net.MalformedURLException - if the URL is not a valid URL

templateDocData

public TemplatedHtmlDocData templateDocData(java.net.URL url)

Create a new HtmlDocData object from the provided url, using this HtmlDocData object as a template. The new HtmlDocData object copies the tags, formats and classes from this template. Classes which subclass this class should implement their own versions of this method to ensure the object returns matches the template.

Parameters:

url - the URL for the document

Returns:

a new HtmlDocData object that uses this object as a template