[prev in list] [next in list] [prev in thread] [next in thread] 

List:       abiword-dev
Subject:    Summarizing articles in a new way
From:       Nadav Rotem <nadavrotem () mail ! ru>
Date:       2003-12-06 14:07:49
[Download RAW message or body]

I started working on a new way of summarizing texts. This time the goal it \
to take a group of articles and join them to a single article; I do so by \
taking one article from the group and replacing parts of it with parts of \
the other articles that have the same signature. 

The code is still hackish , leaks buckets and is NOT ready for any use. but \
it's in the cvs under examples/;

Here (Attached) is a quick demo of an article I joint from various sources;
                                                                            \

> [nadav@gringo news]$ ../join-art cnn.txt abc.txt fox.txt msn.txt \
> usatoday.txt yahoo.txt > ../new_news.txt

new_news: Generated articles
news_replace: a list of replaced blocks

-Nadav Rotem


["new_news.txt" (new_news.txt)]

The storm brought several inches of snow to Virginia and Maryland on \
Friday, and road crews were preparing for a second onslaught with flurries \
overnight expected to turn to heavy snow.   FORECASTERS SAID the system \
could hit even harder over the weekend, with total snow accumulations of up \
to two feet in Massachusetts and near-blizzard conditions in parts of Maine \
and New Hampshire.  These included a blizzard warning for Saturday for the \
New York City area.  Many schoolchildren got an early start to their \
weekend when dozens of schools closed early on Friday.  More than 80 \
fender-benders were reported on the New Jersey Turnpike, where the speed \
limit was reduced from 65 mph to 35 mph because of dangerous conditions.  \
Winds from 20 to 40 mph with higher gusts are forecast Saturday near the \
coast from New York City north to Boston.  "We're hoping the forecasters \
are wrong, but if they're not, we're trusting that people will be staying \
home with their families and off the road so we can get our job done," said \
Anna Farneski, spokeswoman for the Bergen County, N.J.  The warning means \
forecasters expect sustained wind speeds or frequent gusts of 35 mph or \
faster with considerable falling or blowing and drifting snow that will \
drop visibility to zero at times.  "I love storms, but I don't like \
shoveling," Stevenson said.  Forecasters in Massachusetts predicted up to \
18 inches of snow through Sunday.  By Friday evening, The National Weather \
Service had winter storm warnings in effect from Virginia and West Virginia \
northeast into southern New England.  About 100 flights were canceled at \
Philadelphia International Airport on Friday, and some flights were delayed \
at Washington Dulles International Airport.  Flights were canceled at New \
York's La Guardia, New Jersey's Newark Liberty International Airport, \
Boston and Philadelphia airports, and others were delayed by several hours. \
Kennedy airports and New Jersey's Newark Liberty International, according \
to the Port Authority of New York and New Jersey.  The storm had already \
brought a mixture of ice, freezing rain, sleet and snow in mountain regions \
of North Carolina, Virginia and West Virginia on Thursday.  The Weather \
Service said the complex storm system brought in snow and sleet on Friday, \
which changed to rain in some places as temperatures warmed through the \
day.  About 100 flights were canceled at Philadelphia International Airport \
on Friday, and some flights were delayed at Washington Dulles International \
Airport. Cancellations and delays also were expected Saturday at New York's \
La Guardia and John F.  "We're loving it," said Tana Graham, visiting New \
York City from Jackson, Miss.        "The salt is here, the trucks are \
ready, the chains are on, and New York is prepared for the assault," Mayor \
Michael Bloomberg said Friday. "I just bought 100 pounds of ice melt and we \
took out our snow shovels last night.  "The salt is here, the trucks are \
ready, the chains are on, and New York is prepared for the assault," New \
York City Mayor Michael Bloomberg said Friday. At least one death was \
blamed on the storm, that of a 23-year-old man who was killed in an \
accident on a slick road in Virginia.   By Jacqueline Larma, AP  Friday's \
snow was like a warm-up for the main event with even heavier snow and high \
winds making travel even harder on Saturday, forecasters said. Four to 8 \
inches were expected in the Baltimore region.


["news_replace.txt" (news_replace.txt)]

replacing
[Overnight, the heavy snows caused traffic accidents and airport delays.] \
with [  The storm brought several inches of snow to Virginia and Maryland \
on Friday, and road crews were preparing for a second onslaught with \
flurries overnight expected to turn to heavy snow.]

replacing
[ Visibility was down to a quarter-mile in parts of New England as winds \
and snow made driving conditions treacherous.] with [   FORECASTERS SAID \
the system could hit even harder over the weekend, with total snow \
accumulations of up to two feet in Massachusetts and near-blizzard \
conditions in parts of Maine and New Hampshire.]

replacing
[ In New York City a blizzard warning has been issued from 10 a.m. until 10 \
p.m. Saturday.] with [  These included a blizzard warning for Saturday for \
the New York City area.]

replacing
[  In Connecticut, traffic accidents were widespread Friday, with a portion \
of I-95 shut down during rush hour.] with [  Many schoolchildren got an \
early start to their weekend when dozens of schools closed early on \
Friday.]

replacing
[ Authorities urged residents to limit travel Saturday.] with
[  More than 80 fender-benders were reported on the New Jersey Turnpike, \
where the speed limit was reduced from 65 mph to 35 mph because of \
dangerous conditions.]

replacing
[  Winds were expected to be 15-20 mph Saturday, with gusts as high as 40 \
mph.] with [  Winds from 20 to 40 mph with higher gusts are forecast \
Saturday near the coast from New York City north to Boston.]

replacing
[  "We're hoping people take caution," said Julie Bisi, a spokeswoman for \
Connecticut's emergency management office.] with [  "We're hoping the \
forecasters are wrong, but if they're not, we're trusting that people will \
be staying home with their families and off the road so we can get our job \
done," said Anna Farneski, spokeswoman for the Bergen County, N.J.]

replacing
[ "The blowing and drifting could cause problems."] with
[  The warning means forecasters expect sustained wind speeds or frequent \
gusts of 35 mph or faster with considerable falling or blowing and drifting \
snow that will drop visibility to zero at times.]

replacing
[  Forecasters said the height of the storm is expected to come at midday \
Saturday.] with [  "I love storms, but I don't like shoveling," Stevenson \
said.]

replacing
[ Snow totals are predicted to be 10-18 inches.] with
[  Forecasters in Massachusetts predicted up to 18 inches of snow through \
Sunday.]

replacing
[  "The nor'easter has yet to crank up," National Weather Service \
meteorologist Kimberly Buttrick said Friday night from Taunton, \
Massachusetts.] with [  By Friday evening, The National Weather Service had \
winter storm warnings in effect from Virginia and West Virginia northeast \
into southern New England.]

replacing
[  The snow and low clouds were wreaking havoc on air travel in the \
Northeast, with the worst delays for some incoming flights running close to \
15 hours.] with [  About 100 flights were canceled at Philadelphia \
International Airport on Friday, and some flights were delayed at \
Washington Dulles International Airport.]

replacing
[ Ground stops were in effect late Friday at Newark International Airport \
in New Jersey, Philadelphia International Airport in Pennsylvania, and \
Cleveland Hopkins International Airport in Ohio.] with [  Flights were \
canceled at New York's La Guardia, New Jersey's Newark Liberty \
International Airport, Boston and Philadelphia airports, and others were \
delayed by several hours.]

replacing
[  A ground stop means traffic at other airports bound for that destination \
is not allowed to depart.] with [ Kennedy airports and New Jersey's Newark \
Liberty International, according to the Port Authority of New York and New \
Jersey.]

replacing
[  Winter storm warnings were in effect from eastern Ohio to Connecticut \
and from West Virginia into Massachusetts.] with [  The storm had already \
brought a mixture of ice, freezing rain, sleet and snow in mountain regions \
of North Carolina, Virginia and West Virginia on Thursday.]

replacing
[ Freezing rain, sleet and snow from the first storm blanketed parts of \
North Carolina, Virginia and West Virginia and more is expected as the \
storm moves up the Atlantic coast.] with [  The Weather Service said the \
complex storm system brought in snow and sleet on Friday, which changed to \
rain in some places as temperatures warmed through the day.]

replacing
[  Washington is expected to get 4-6 inches, and Philadelphia is bracing \
for 8-10 inches.] with [  About 100 flights were canceled at Philadelphia \
International Airport on Friday, and some flights were delayed at \
Washington Dulles International Airport.]

replacing
[  New York is expected to get at least 8-14 inches of snow.] with
[ Cancellations and delays also were expected Saturday at New York's La \
Guardia and John F.]

replacing
[ New York City has 200,000 tons of salt to be dispersed on roads, and \
1,700 snowplows at the ready.] with [  "We're loving it," said Tana Graham, \
visiting New York City from Jackson, Miss.]

replacing
[  New York City Mayor Michael Bloomberg said 3,000 city workers would be \
on duty to make sure the city's 6,000 miles of roads stay clear.] with [    \
"The salt is here, the trucks are ready, the chains are on, and New York is \
prepared for the assault," Mayor Michael Bloomberg said Friday.]

replacing
[  "This is snow that we know how to take care of;] with
[ "I just bought 100 pounds of ice melt and we took out our snow shovels \
last night.]

replacing
[ they've got the right equipment," he said in a news conference Friday \
night.] with [  "The salt is here, the trucks are ready, the chains are on, \
and New York is prepared for the assault," New York City Mayor Michael \
Bloomberg said Friday.]

replacing
[  Bloomberg said the roads were fine, but when the storm blows in \
Saturday, conditions could be bad.] with [ At least one death was blamed on \
the storm, that of a 23-year-old man who was killed in an accident on a \
slick road in Virginia.]

replacing
[  "The winds are forecast to be very strong, really blizzard conditions, \
that's when you'll have to worry about trees coming down, power lines," he \
said.] with [   By Jacqueline Larma, AP  Friday's snow was like a warm-up \
for the main event with even heavier snow and high winds making travel even \
harder on Saturday, forecasters said.]

replacing
[  The weather grounded President Bush's planned Marine One helicopter \
flight to a re-election campaign luncheon in Baltimore, Maryland, where 6 \
inches of snow by midmorning had clogged streets and slowed traffic.] with \
[ Four to 8 inches were expected in the Baltimore region.]



[prev in list] [next in list] [prev in thread] [next in thread]