The OpenNET Project / Index page

[ новости /+++ | форум | теги | ]

Интерактивная система просмотра системных руководств (man-ов)

 ТемаНаборКатегория 
 
 [Cписок руководств | Печать]

HTML::TokeParser::Simple (3)
  • >> HTML::TokeParser::Simple (3) ( Разные man: Библиотечные вызовы )
  •  

    NAME

    HTML::TokeParser::Simple - easy to use HTML::TokeParser interface
     
    

    SYNOPSIS

     use HTML::TokeParser::Simple;
     my $p = HTML::TokeParser::Simple->new( $somefile );
    
    

     while ( my $token = $p->get_token ) {
         # This prints all text in an HTML doc (i.e., it strips the HTML)
         next unless $token->is_text;
         print $token->as_is;
     }
    
    
     

    DESCRIPTION

    "HTML::TokeParser" is a fairly common method of parsing HTML. However, the tokens returned are not exactly intuitive to parse:

     ["S",  $tag, $attr, $attrseq, $text]
     ["E",  $tag, $text]
     ["T",  $text, $is_data]
     ["C",  $text]
     ["D",  $text]
     ["PI", $token0, $text]
    
    

    To simplify this, "HTML::TokeParser::Simple" allows the user ask more intuitive (read: more self-documenting) questions about the tokens returned. Specifically, there are 7 "is_foo" type methods and 5 "return_bar" type methods. The "is_" methods allow you to determine the token type and the "return_" methods get the data that you need.

    You can also rebuild some tags on the fly. Frequently, the attributes associated with start tags need to be altered, added to, or deleted. This functionality is built in.

    Since this is a subclass of "HTML::TokeParser", all "HTML::TokeParser" methods are available. To truly appreciate the power of this module, please read the documentation for "HTML::TokeParser" and "HTML::Parser".

    The following will be brief descriptions of the available methods followed by examples.  

    is_ Methods

    * is_start_tag([$tag])
    Use this to determine if you have a start tag. An optional ``tag type'' may be passed. This will allow you to match if it's a particular start tag. The supplied tag is case-insensitive.

     if ( $token->is_start_tag( 'font' ) ) { ... }
    
    

    Optionally, you may pass a regular expression as an argument. To match all header (h1, h2, ... h6) tags:

     if ( $token->is_start_tag( qr/^h[123456]$/ ) ) { ... }
    
    
    * is_end_tag([$tag])
    Use this to determine if you have an end tag. An optional ``tag type'' may be passed. This will allow you to match if it's a particular end tag. The supplied tag is case-insensitive.

    When testing for an end tag, the forward slash on the tag is optional.

     while ( $token = $p->get_token ) {
       if ( $token->is_end_tag( 'form' ) ) { ... }
     }
    
    

    Or:

     while ( $token = $p->get_token ) {
       if ( $token->is_end_tag( '/form' ) ) { ... }
     }
    
    

    Optionally, you may pass a regular expression as an argument.

    * is_tag([$tag])
    Use this to determine if you have any tag. An optional ``tag type'' may be passed. This will allow you to match if it's a particular tag. The supplied tag is case-insensitive.

     if ( $token->is_tag ) { ... }
    
    

    Optionally, you may pass a regular expression as an argument.

    * is_text()
    Use this to determine if you have text. Note that this is not to be confused with the "return_text" (deprecated) method described below! "is_text" will identify text that the user typically sees display in the Web browser.
    * is_comment()
    Are you still reading this? Nobody reads POD. Don't you know you're supposed to go to CLPM, ask a question that's answered in the POD and get flamed? It's a rite of passage.

    Really.

    "is_comment" is used to identify comments. See the HTML::Parser documentation for more information about comments. There's more than you might think.

    * is_declaration()
    This will match the DTD at the top of your HTML. (You do use DTD's, don't you?)
    * is_process_instruction()
    Process Instructions are from XML. This is very handy if you need to parse out PHP and similar things with a parser.
     

    The return_ methods

     

    Note:

    In case it's not blindingly obvious (I've been bitten by this myself when writing the tests), you should generally test what type of token you have before you call some "return_" methods. For example, if you have an end tag, there is no point in calling the "return_attrseq" method. Calling an innapropriate method will return an empty string.

    As noted for the "is_" methods, these methods are case-insensitive after the "return_" part.

    * return_tag()
    Do you have a start tag or end tag? This will return the type (lower case).
    * return_attr([$attribute])
    If you have a start tag, this will return a hash ref with the attribute names as keys and the values as the values.

    If you pass in an attribute name, it will return the value for just that attribute. Returns "undef" if the attribute is not found.

    * return_attrseq()
    For a start tag, this is an array reference with the sequence of the attributes, if any.
    * return_text()
    This method has been deprecated in favor of "as_is". Programmers were getting confused over the difference between "is_text", "return_text", and some parser methods such as "HTML::TokeParser::get_text" and friends. This confusion stems from the fact that your author is a blithering idiot when it comes to choosing methods names :)

    Using this method still succeeds, but will now carp.

    * as_is()
    This is the exact text of whatever the token is representing.
    * return_token0()
    For processing instructions, this will return the token found immediately after the opening tag. Example: For <?php, ``php'' will be the start of the returned string.
     

    Tag munging methods

    The "delete_attr()" and "set_attr()" methods allow the programmer to rewrite tag attributes on the fly. It should be noted that bad HTML will be ``corrected'' by this. Specifically, the new tag will have all attributes lower-cased with the values properly quoted.

    Self-closing tags (e.g. <hr />) are also handled correctly. Some older browsers require a space prior to the final slash in a self-closed tag. If such a space is detected in the original HTML, it will be preserved.

    * delete_attr($name)
    This method attempts to delete the attribute specified. It will "croak" if called on anything other than a start tag. The argument is case-insensitive, but must otherwise be an exact match of the attribute you are attempting to delete. If the attribute is not found, the method will return without changing the tag.

     # <body bgcolor="#FFFFFF">
     $token->delete_attr('bgcolor');
     print $token->as_is;
     # <body>
    
    

    After this method is called, if successful, the "as_is()", "return_attr()" and "return_attrseq()" methods will all return updated results.

    * set_attr($name,$value)
    This method will set the value of an attribute. If the attribute is not found, then "return_attrseq()" will have the new attribute listed at the end. Two arguments

     # <p>
     $token->set_attr('class','some_class');
     print $token->as_is;
     # <p class="some_class">
    
    

     # <body bgcolor="#FFFFFF">
     $token->set_attr('bgcolor','red');
     print $token->as_is;
     # <body bgcolor="red">
    
    

    After this method is called, if successful, the "as_is()", "return_attr()" and "return_attrseq()" methods will all return updated results.

    * rewrite_tag()
    This method rewrites the tag. The tag name and the name of all attributes will be lower-cased. Values that are not quoted with double quotes will be. This may be called on both start or end tags. Note that both "set_attr()" and "delete_attr()" call this method prior to returning.

    If called on a token that is not a tag, it simply returns. Regardless of how it is called, it returns the token.

     # <body alink=#0000ff BGCOLOR=#ffffff class='none'>
     $token->rewrite_tag;
     print $token->as_is;
     # <body alink="#0000ff" bgcolor="#ffffff" class="none">
    
    

    A quick cleanup of sloppy HTML is now the following:

     my $parser = HTML::TokeParser::Simple->new( $ugly_html );
     while (my $token = $parser->get_token) {
         $token->rewrite_tag;
         print $token->as_is;
     }
    
    
     

    Important note:

    Some people get confused and try to call parser methods on tokens and token methods (those described above) on methods. To prevent this, "HTML::TokeParser::Simple" versions 1.4 and above now bless all tokens into a new class which inherits nothing. Please keep this in mind while using this module (and many thanks to PodMaster <http://www.perlmonks.org/index.pl?node_id=107642> for pointing out this issue to me.  

    Examples

     

    Finding comments

    For some strange reason, your Pointy-Haired Boss (PHB) is convinced that the graphics department is making fun of him by embedding rude things about him in HTML comments. You need to get all HTML comments from the HTML.

     use strict;
     use HTML::TokeParser::Simple;
    
    

     my @html_docs = glob( "*.html" );
    
    

     open PHB, "> phbreport.txt" or die "Cannot open phbreport for writing: $!";
    
    

     foreach my $doc ( @html_docs ) {
         print "Processing $doc\n";
         my $p = HTML::TokeParser::Simple->new( $doc );
         while ( my $token = $p->get_token ) {
             next unless $token->is_comment;
             print PHB $token->as_is, "\n";
         }
     }
    
    

     close PHB;
    
    
     

    Stripping Comments

    Uh oh. Turns out that your PHB was right for a change. Many of the comments in the HTML weren't very polite. Since your entire graphics department was just fired, it falls on you need to strip those comments from the HTML.

     use strict;
     use HTML::TokeParser::Simple;
    
    

     my $new_folder = 'no_comment/';
     my @html_docs  = glob( "*.html" );
    
    

     foreach my $doc ( @html_docs ) {
         print "Processing $doc\n";
         my $new_file = "$new_folder$doc";
    
    

         open PHB, "> $new_file" or die "Cannot open $new_file for writing: $!";
    
    

         my $p = HTML::TokeParser::Simple->new( $doc );
         while ( my $token = $p->get_token ) {
             next if $token->is_comment;
             print PHB $token->as_is;
         }
         close PHB;
     }
    
    
     

    Changing form tags

    Your company was foo.com and now is bar.com. Unfortunately, whoever wrote your HTML decided to hardcode ``http://www.foo.com/'' into the "action" attribute of the form tags. You need to change it to ``http://www.bar.com/''.

     use strict;
     use HTML::TokeParser::Simple;
    
    

     my $new_folder = 'new_html/';
     my @html_docs  = glob( "*.html" );
    
    

     foreach my $doc ( @html_docs ) {
         print "Processing $doc\n";
         my $new_file = "$new_folder$doc";
    
    

         open FILE, "> $new_file" or die "Cannot open $new_file for writing: $!";
    
    

         my $p = HTML::TokeParser::Simple->new( $doc );
         while ( my $token = $p->get_token ) {
             if ( $token->is_start_tag('form') ) {
                 my $action = $token->return_attr->{action};
                 $action =~ s/www\.foo\.com/www.bar.com/;
                 $token->set_attr('action', $action);
             }
             print FILE $token->as_is;
         }
         close FILE;
     }
    
    
     

    COPYRIGHT

    Copyright (c) 2001 Curtis ``Ovid'' Poe. All rights reserved. This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself  

    AUTHOR

    Curtis ``Ovid'' Poe poec@yahoo.com  

    BUGS

    Use of $HTML::Parser::VERSION which is less than 3.25 may result in incorrect behavior as older versions do not always handle XHTML correctly. It is the programmer's responsibility to verify that the behavior of this code matches the programmer's needs.

    Note that "HTML::Parser" processes text in 512 byte chunks. This sometimes will cause strange behavior and cause text to be broken into more than one token. You can suppress this behavior with the following command:

     $p->unbroken_text( [$bool] );
    
    

    See the "HTML::Parser" documentation and http://www.perlmonks.org/index.pl?node_id=230667 for more information.

    Address bug reports and comments to: poec@yahoo.com. When sending bug reports, please provide the version of "HTML::Parser", "HTML::TokeParser", "HTML::TokeParser::Simple", the version of Perl, and the version of the operating system you are using.


     

    Index

    NAME
    SYNOPSIS
    DESCRIPTION
    is_ Methods
    The return_ methods
    Note:
    Tag munging methods
    Important note:
    Examples
    Finding comments
    Stripping Comments
    Changing form tags
    COPYRIGHT
    AUTHOR
    BUGS


    Поиск по тексту MAN-ов: 




    Партнёры:
    PostgresPro
    Inferno Solutions
    Hosting by Hoster.ru
    Хостинг:

    Закладки на сайте
    Проследить за страницей
    Created 1996-2024 by Maxim Chirkov
    Добавить, Поддержать, Вебмастеру