Top Links: >> 80. Technology >> Internet Technology Summit Program >> 3. Threads and Network >> 3.2. Network

Current Topic: 3.2.6. Web page reader

Web page reader example

Here is the example of the WebPageReader class with its method readWebPage().
In the example we create a URL object from a URL string, such as "http://gmail.com".

After creating a URL object we use its openStream() method to create an InputStream. Then we engage a usual scenario of using Input streams by creating a proper input stream for proper type of information.

In this case to read a web page, which is html/text by its nature, we create a BufferedReader stream out of InputStream.

Then we read line by line in a loop and add each line to our html string.

package its.day16.http;



import java.io.BufferedReader;

import java.io.InputStreamReader;

import java.net.URL;

/**

 * The WebReader reads web pages base on provided URL

 * @author Jeff.Zhuk@Javaschool.com

 *

 */

public class WebReader {

	/**

	 * The readWebPage() method reads a web page based on a provided URL

	 * @param urlString

	 * @return html web page

	 * @throws Exception

	 */

	public String readWebPage(String urlString) throws Exception {

		URL url = new URL(urlString);

		String html = "";

		BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));        

        for (String inputLine = null; (inputLine = in.readLine()) != null; ) {

            html += inputLine + System.lineSeparator();

        }

        in.close();

        return html;

	}

	/**

	 * The main() method will test the 

	 * @param args

	 * @throws Exception

	 */

	public static void main(String[] args) throws Exception {

		WebReader webReader = new WebReader();

		String html = webReader.readWebPage("http://gmail.com");

		System.out.println(html);

	}

}

Was it clear so far?

Assignments:
1. Open Eclipse and navigate to the project week7.network.
2. Open the project to see the src directory.
3. Right click on the src directory – NEW – Package its.day16.http
4. Right click on this new package and create – NEW – Class - WebPageReader
5. Type the source above into the class.
6. Right mouse click on the source – Run – As Java Application
7. You will see in the Console an html source of the target page.
8. Change the target page to another web URL, starting with http://.
9. Run this application again and see the result in the Console.

<br/>package its.day16.http;
<br/>
<br/>import java.io.BufferedReader;
<br/>import java.io.InputStreamReader;
<br/>import java.net.URL;
<br/>/**
<br/> * The WebReader reads web pages base on provided URL
<br/> * @author Jeff.Zhuk@Javaschool.com
<br/> *
<br/> */
<br/>public class WebReader {
<br/>	/**
<br/>	 * The readWebPage() method reads a web page based on a provided URL
<br/>	 * @param urlString
<br/>	 * @return html web page
<br/>	 * @throws Exception
<br/>	 */
<br/>	public String readWebPage(String urlString) throws Exception {
<br/>		URL url = new URL(urlString);
<br/>		String html = "";
<br/>		BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));        
<br/>        for (String inputLine = null; (inputLine = in.readLine()) != null; ) {
<br/>            html += inputLine + System.lineSeparator();
<br/>        }
<br/>        in.close();
<br/>        return html;
<br/>	}
<br/>	/**
<br/>	 * The main() method will test the 
<br/>	 * @param args
<br/>	 * @throws Exception
<br/>	 */
<br/>	public static void main(String[] args) throws Exception {
<br/>		WebReader webReader = new WebReader();
<br/>		String html = webReader.readWebPage("http://gmail.com");
<br/>		System.out.println(html);
<br/>	}
<br/>}
<br/>

Was it clear so far?
onclick="window.location.href='/BASE/jsp/demo.jsp?checkFlavor=itsp&issueID=74&intro=general&group=aitu&ur=f'">

Have a suggestion? - shoot an email
Looking for something special? - Talk to me
Read: IT of the future: AI and Semantic Cloud Architecture | Fixing Education
Do you want to move from theory to practice and become a magician? Learn and work with us at Internet Technology University (ITU) - JavaSchool.com.

Technology that we offer and How this works: English | Spanish | Russian | French

Internet Technology University | JavaSchool.com | Copyrights © Since 1997 | All Rights Reserved
Patents: US10956676, US7032006, US7774751, US7966093, US8051026, US8863234
Including conversational semantic decision support systems (CSDS) and bringing us closer to The message from 2040
Privacy Policy