Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ All notable changes to this project will be documented in this file.
* **Early RIPE RIS MRT support**: Parse deprecated MRT Type 5 BGP OPEN, UPDATE, NOTIFY, KEEPALIVE, and STATE_CHANGE records, along with historical TABLE_DUMP v1 records that batch multiple entries and declare their physical length four bytes short. Record iteration preserves each physical TABLE_DUMP batch while element, update, and route iteration expands its entries.
* **Historical RIPE regression fixtures**: Added original RRC00 update and bview gzip files from 1999 and January 2000 as repository-only, offline integration fixtures.
* **RFC 10005 Link Bandwidth Extended Community**: Typed parsing and encoding for the BGP Link Bandwidth Extended Community in both transitive (`0x00`) and non-transitive (`0x40`) forms ([#299](https://github.com/bgpkit/bgpkit-parser/issues/299)). Exposes the Global Administrator, bandwidth in bytes per second, and transitivity, and preserves the wire type on encode.
* **Cisco `sh ip bgp` text dump parsing** ([#320](https://github.com/bgpkit/bgpkit-parser/issues/320)): new `parser::text_dump` module parses fixed-width text RIB dumps published by PCH (daily routing table snapshots) and route-views (`oix-full-snapshot-*.bz2`) into `BgpElem`s. Column offsets are derived from the table header so blank numeric columns (Metric, LocPrf, Weight) stay distinct from AS-path data; multipath continuation lines and wrapped prefixes (including IPv6) are supported. Route-views dumps without the `BGP table version` / `local AS` preamble parse with sentinel peer identity (`0.0.0.0` / AS0). Also provides `detect_text_dump` for format sniffing and `infer_timestamp_from_path` for PCH (`YYYY.MM.DD`) and route-views (`YYYY-MM-DD-HHMM`) file-name timestamps. Ported from monocle ([#143](https://github.com/bgpkit/monocle/pull/143), [#146](https://github.com/bgpkit/monocle/pull/146)).
* **Unified MRT/text-dump parser API**: `BgpkitParser` gains three new constructor groups that integrate text dumps into the standard `for elem in parser` iteration loop. `new_text(path)` / `from_text_reader(r)` parse a known text dump; `new_auto(path)` / `from_auto_reader(r)` peek the first bytes and auto-dispatch to the text or MRT path. Both text-dump paths **stream elements lazily** — one route line at a time, constant memory — via a new `TextDumpElemIterator`. All existing filter methods (`add_filter`, `with_filters`, etc.) work on both paths. The default `new(path)` constructor remains MRT-only.
* **RFC 5543 Traffic Engineering attribute**: Typed parsing and encoding for the BGP Traffic Engineering attribute (type 24) ([#290](https://github.com/bgpkit/bgpkit-parser/issues/290)). Exposes the Switching Capability, Encoding, Reserved, and eight Maximum LSP Bandwidth fields (IEEE-754), and retains switching-capability-specific information as raw bytes with wire-faithful round-trip.

### Fixed
Expand Down
28 changes: 28 additions & 0 deletions examples/parse_text_dump_auto.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
use bgpkit_parser::BgpkitParser;

/// This example demonstrates `BgpkitParser::new_auto`, which auto-detects
/// whether the input is an MRT file or a Cisco `sh ip bgp` text dump and
/// parses accordingly — no need to know the format in advance.
///
/// The same `for elem in parser` loop works for both MRT and text dumps.
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

// A Cisco `sh ip bgp` text dump from PCH — new_auto detects and parses it.
let text_url = "https://downloads.pch.net/files/Routing_Data/IPv4_daily_snapshots/2026/07/route-collector.bom2.pch.net/route-collector.bom2.pch.net-ipv4_bgp_routes.2026.07.01.gz";
log::info!("auto-detecting and parsing text dump: {text_url}");
let count = BgpkitParser::new_auto(text_url)
.unwrap()
.into_elem_iter()
.count();
log::info!("text dump: {count} elements");

// A standard MRT RIB file — new_auto treats it as MRT (lazy streaming).
let mrt_url = "https://spaces.bgpkit.org/parser/update-example.gz";
log::info!("auto-detecting and parsing MRT file: {mrt_url}");
let count = BgpkitParser::new_auto(mrt_url)
.unwrap()
.into_elem_iter()
.count();
log::info!("MRT file: {count} elements");
}
26 changes: 26 additions & 0 deletions examples/parse_text_dump_pch.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
use bgpkit_parser::BgpkitParser;

/// This example parses a PCH daily routing table snapshot, which is a Cisco
/// `sh ip bgp` fixed-width text dump (gzip-compressed), into `BgpElem`s.
///
/// `BgpkitParser::new_text` auto-detects decompression, infers the snapshot
/// timestamp from the file name, and returns elements through the same
/// `for elem in parser` interface used for MRT files.
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

let url = "https://downloads.pch.net/files/Routing_Data/IPv4_daily_snapshots/2026/07/route-collector.bom2.pch.net/route-collector.bom2.pch.net-ipv4_bgp_routes.2026.07.01.gz";

log::info!("opening {url}");
let parser = BgpkitParser::new_text(url).unwrap();
log::info!("parsing text dump");

let mut count = 0;
for elem in parser {
if count < 5 {
println!("{elem}");
}
count += 1;
}
log::info!("parsed {count} elements");
}
28 changes: 28 additions & 0 deletions examples/parse_text_dump_routeviews.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
use bgpkit_parser::BgpkitParser;

/// This example parses a route-views `sh ip bgp` snapshot
/// (`oix-full-snapshot-*.bz2`) into `BgpElem`s.
///
/// `BgpkitParser::new_text` handles the bzip2 decompression and timestamp
/// inference, and streams elements one route line at a time (constant memory).
///
/// Route-views snapshots omit the `BGP table version` / `local AS` preamble,
/// so parsed elements carry the sentinel peer identity `0.0.0.0` / AS0.
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

let url = "https://archive.routeviews.org/oix-route-views/2026.07/oix-full-snapshot-2026-07-01-0000.bz2";

log::info!("opening {url}");
let parser = BgpkitParser::new_text(url).unwrap();
log::info!("streaming text dump");

let mut count = 0;
for elem in parser {
if count < 5 {
println!("{elem}");
}
count += 1;
}
log::info!("parsed {count} elements");
}
15 changes: 15 additions & 0 deletions src/parser/iters/default.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ impl<R: Read> Iterator for RecordIterator<R> {
type Item = MrtRecord;

fn next(&mut self) -> Option<MrtRecord> {
// Text-dump parsers have no MRT-record representation; short-circuit
// instead of spinning forever on Unsupported errors from next_record().
if self.parser.text_dump_iter.is_some() {
return None;
}
self.count += 1;
loop {
return match self.parser.next_record() {
Expand Down Expand Up @@ -139,6 +144,16 @@ impl<R: Read> Iterator for ElemIterator<R> {
self.count += 1;

loop {
// Fast path: drain streaming text-dump elems directly, with filter support.
if let Some(iter) = &mut self.record_iter.parser.text_dump_iter {
for elem in iter.by_ref() {
if elem.match_filters(&self.record_iter.parser.filters) {
return Some(elem);
}
}
return None;
}

if self.cache_elems.is_empty() {
// refill cache elems
loop {
Expand Down
15 changes: 15 additions & 0 deletions src/parser/iters/fallible.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ impl<R: Read> Iterator for FallibleRecordIterator<R> {
type Item = Result<MrtRecord, ParserErrorWithBytes>;

fn next(&mut self) -> Option<Self::Item> {
// Text-dump parsers have no MRT-record representation; short-circuit
// instead of repeatedly returning Unsupported errors from next_record().
if self.parser.text_dump_iter.is_some() {
return None;
}
loop {
match self.parser.next_record() {
Ok(record) => {
Expand Down Expand Up @@ -96,6 +101,16 @@ impl<R: Read> Iterator for FallibleElemIterator<R> {

fn next(&mut self) -> Option<Self::Item> {
loop {
// Fast path: drain streaming text-dump elems directly, with filter support.
if let Some(iter) = &mut self.record_iter.parser.text_dump_iter {
for elem in iter.by_ref() {
if elem.match_filters(&self.record_iter.parser.filters) {
return Some(Ok(elem));
}
}
return None;
}

// First check if we have cached elements
if !self.cache_elems.is_empty() {
if let Some(elem) = self.cache_elems.pop() {
Expand Down
4 changes: 4 additions & 0 deletions src/parser/iters/raw.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ impl<R: Read> Iterator for RawRecordIterator<R> {
type Item = RawMrtRecord;

fn next(&mut self) -> Option<RawMrtRecord> {
// Text-dump parsers have no MRT-record representation; short-circuit.
if self.parser.text_dump_iter.is_some() {
return None;
}
self.count += 1;
loop {
match chunk_mrt_record(&mut self.parser.reader) {
Expand Down
8 changes: 8 additions & 0 deletions src/parser/iters/route.rs
Original file line number Diff line number Diff line change
Expand Up @@ -681,6 +681,10 @@ impl<R: Read> Iterator for RouteIterator<R> {
type Item = BgpRouteElem;

fn next(&mut self) -> Option<Self::Item> {
// Text-dump parsers have no MRT-record representation; short-circuit.
if self.parser.text_dump_iter.is_some() {
return None;
}
loop {
match self.pending_routes.next_route() {
Ok(Some(route)) => {
Expand Down Expand Up @@ -789,6 +793,10 @@ impl<R: Read> Iterator for FallibleRouteIterator<R> {
type Item = Result<BgpRouteElem, ParserErrorWithBytes>;

fn next(&mut self) -> Option<Self::Item> {
// Text-dump parsers have no MRT-record representation; short-circuit.
if self.parser.text_dump_iter.is_some() {
return None;
}
loop {
match self.pending_routes.next_route() {
Ok(Some(route)) => {
Expand Down
9 changes: 9 additions & 0 deletions src/parser/iters/update.rs
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ impl<R: Read> Iterator for UpdateIterator<R> {
type Item = MrtUpdate;

fn next(&mut self) -> Option<MrtUpdate> {
// Text-dump parsers have no MRT-record representation; short-circuit
// instead of spinning forever on Unsupported errors from next_record().
if self.parser.text_dump_iter.is_some() {
return None;
}
loop {
if let Some(message) = self.pending_table_dump.pop() {
return Some(MrtUpdate::TableDumpMessage(message));
Expand Down Expand Up @@ -316,6 +321,10 @@ impl<R: Read> Iterator for FallibleUpdateIterator<R> {
type Item = Result<MrtUpdate, crate::error::ParserErrorWithBytes>;

fn next(&mut self) -> Option<Self::Item> {
// Text-dump parsers have no MRT-record representation; short-circuit.
if self.parser.text_dump_iter.is_some() {
return None;
}
loop {
if let Some(message) = self.pending_table_dump.pop() {
return Some(Ok(MrtUpdate::TableDumpMessage(message)));
Expand Down
Loading
Loading