Tech Explorer

Problem

Technical discovery is fragmented across GitHub Trending, OSS Insight, Awesome lists, and paper feeds, making manual curation slow and inconsistent. tech-explorer aggregates these sources into reviewable reports, reducing repeated scraping, field alignment, and formatting work.

How It Works

• Trending: fetch_real_trending.py scrapes github.com/trending for the real Top 10, preserving fields such as stars, forks, and new stars.
• OSS Insight: fetch_github_trending.py uses the TiDB API to rank projects by a composite score of stars, forks, PRs, and pushes, falling back to GitHub Search when unavailable.
• AI Top: fetch_top_starred.py searches active, high-star projects by topic, such as ai, llm, or machine-learning.
• Awesome: fetch_awesome_list.py searches awesome-topic repositories and parses project names, links, and descriptions from READMEs.
• HF Papers: fetch_hf_papers.py retrieves popular Hugging Face papers, including upvotes, arXiv IDs, and repository links.
• DeepWiki: generates a deep research report for a chosen repository, optionally supplemented by Baidu search for Chinese-context information.

The workflow can also produce a styled Excel workbook with openpyxl, translate English descriptions into Chinese, save reports to a draft directory, and update the document index file.

Boundaries

It is useful for building a monthly tech radar, shortlisting AI open-source projects, and reviewing recent papers. It depends on external pages, APIs, and network conditions, so output may be affected by rate limits, proxies, or site changes. Human review is still required to assess maturity, maintenance status, and business fit.