Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DomainFrom ¶
func GetHashIDs ¶
Helper function to extract IDs from links
func SafeUrlParser ¶
func URLToPathSlug ¶
URLToPathSlug returns the path part of a URL (without leading/trailing slashes). If input is already a path like "/tag/slug-here/" or "slug-here", it normalizes and returns it.
Types ¶
type Link ¶
type Link struct {
Hash string `json:"hash" gorm:"unique"`
Domain string `json:"domain"`
SitemapIndex string `json:"sitemap_index"`
Url string `json:"url"`
}
type Site ¶
type Site struct {
Domain string `json:"domain" gorm:"unique"`
Url string `json:"url"`
Sitemaps string `json:"sitemaps" gorm:"type:text"` //sitemap indexs
CreatedAt int64 `json:"created_at" gorm:"autoCreateTime"`
UpdatedAt int64 `json:"updated_at" gorm:"autoUpdateTime"`
}
func (*Site) ParseSitemaps ¶
func (*Site) SetSitemaps ¶
type SitemapIndex ¶
type SitemapIndex struct {
Hash string `json:"hash" gorm:"unique"`
Domain string `json:"domain"`
Url string `json:"url"`
Scanned bool `json:"scanned"`
}
func NewSitemapIndex ¶
func NewSitemapIndex(url string) *SitemapIndex
func NewSitemapIndexsFrom ¶
func NewSitemapIndexsFrom(site *Site) []SitemapIndex
func (SitemapIndex) HashID ¶
func (s SitemapIndex) HashID() string
Click to show internal directories.
Click to hide internal directories.