IterableDataset

[yueyinqiu]

It seems that PyTorch has something called IterableDataset, which is different from IterableDataset in TorchSharp.

[alinpahontu2912]

@yueyinqiu Thank you for reporting the issue regarding IterableDataset. We appreciate your input! It’s important to clarify the differences between IterableDataset in PyTorch and TorchSharp.

I noticed that there are linked pull requests (#1354 and #1411) related to concatenating datasets. Did these PRs address your concerns, or do you feel that the issue remains unresolved?

As TorchSharp has evolved significantly over the past couple of years, your insights would be invaluable in determining whether this issue is still relevant. If it is, could you please provide any updated information or specific examples? Thank you again for your contribution!

[yueyinqiu]

It’s great to see TorchSharp getting active again! Is the team planning to resume active maintenance and development? Many of us in the community would be thrilled to hear that.

Regarding the dataset issues: as I recall, #1354 was later reverted for some reason. Also, #1411 did not remove IterableDataset because we were trying to avoid too many breaking changes in that PR. If my memory serves me right, once #1411 is merged, IterableDataset could potentially be removed since users can transition to the new interfaces instead.

However, since that PR is quite old, I can't guarantee it will still work perfectly with the current codebase. If we're resuming active development, I'd be happy to spend some time revisiting and updating it.

By the way, I’d also like to ask about the current policy on breaking changes. Previously, we were generally open to them—is that still the case, or should we be more conservative now? And also, are we still targeting at .NET Framework / Standard as well? This will affect the design and implementation of some APIs. (Our original idea was to support all .NET versions that had not yet reached End Of Support, and .NET Framework 4.7-4.8 do not have such End Of Support date, although they were actually quite outdated.)