[Solved] Unable to scrape data from Internet using Android intents

by


Have you considered using an http framework for Android instead? It’s a lot less code and also runs the requests in the background. This example uses the loopj async client

build.gradle:

compile 'com.loopj.android:android-async-http:1.4.9'
compile 'cz.msebera.android:httpclient:4.4.1.2'

Test code:

@Test
public void parseHttp() throws Exception {

    AsyncHttpClient client = new AsyncHttpClient();
    final CountDownLatch latch = new CountDownLatch(1);

    String url = "http://stackoverflow.com/questions/38959381/unable-to-scrape-data-from-internet-using-android-intents";

    client.get(url, new AsyncHttpResponseHandler(Looper.getMainLooper()) {
        @Override
        public void onSuccess(int statusCode, Header[] headers, byte[] responseBody) {
            String body = new String(responseBody);
            Pattern p = Pattern.compile("<h1(.*)<\\/h1>");
            Matcher m = p.matcher(body);
            Log.d("tag", "success");
            if ( m.find() ) {
                String match = m.group(1);
                Log.d("tag", match);
            }
            latch.countDown();
        }

        @Override
        public void onFailure(int statusCode, Header[] headers, byte[] responseBody, Throwable error) {

            Log.d("tag", "failure");
            latch.countDown();
        }
    });

    latch.await();
}

Update:

Added additional import above: compile 'cz.msebera.android:httpclient:4.4.1.2'

import cz.msebera.android.httpclient.Header;
import cz.msebera.android.httpclient.HttpRequest;
import cz.msebera.android.httpclient.HttpResponse;
import cz.msebera.android.httpclient.HttpStatus;

3

solved Unable to scrape data from Internet using Android intents