bc2caadb74
* Implement webhook events for external integrations (#574) * Implement webhook events for external integrations Reference #556 * move message type to models and remove duplicate * add json header so content type can be determined * Pass at migrating webhooks to datastore + management apis (#589) * Pass at migrating webhooks to datastore + management apis * Support nil lastUsed timestamps and return back the new webhook on create * Cleanup from review feedback * Simplify a bit Co-authored-by: Aaron Ogle <aaron@geekgonecrazy.com> Co-authored-by: Gabe Kangas <gabek@real-ity.com> * Webhook query cleanup * Access tokens + Send system message external API (#585) * New add, get and delete access token APIs * Create auth token middleware * Update last_used timestamp when using an access token * Add auth'ed endpoint for sending system messages * Cleanup * Update api spec for new apis * Commit updated API documentation * Add auth'ed endpoint for sending user chat messages * Return access token string * Commit updated API documentation * Fix route * Support nil lastUsed time * Commit updated Javascript packages * Remove duplicate function post rebase * Fix msg id generation * Update controllers/admin/chat.go Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com> * Webhook query cleanup * Add SystemMessageSent to EventType Co-authored-by: Owncast <owncast@owncast.online> Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com> * Set webhook as used on completion. Closes #610 * Display webhook errors as errors * Commit updated API documentation * Add user joined chat event * Change integration API paths. Update API spec * Update development version of admin that supports integration apis * Commit updated API documentation * Add automated tests for external integration APIs * check error * quiet this test for now * Route up some additional 3rd party apis. #638 * Commit updated API documentation * Save username on user joined event * Add missing scope to valid scopes list * Add generic chat action event API for 3rd parties. Closes #666 * Commit updated API documentation * First pass at moving WIP config framework into project for #234 * Only support exported fields in custom types * Using YP get/set key as a first pass at using the data layer. Fixes + integration. * Ignore test db * Start adding getters and setters for config values * More get/set config work. Starting to populate api with data * Wire up some config edit endpoints * More endpoints * Disable cors middleware * Add more endpoints and add test to test them * Remove the in-memory change APIs * Add endpoint for changing tags * Add more config endpoints * Starting to point more things away from config file and to the datastore * Populate YP with db data * Create new util method for parsing page body markdown and return it in api * Verify proposed path to ffmpeg * For development purposes show the config key in logs * Move stats values to datastore * Moving over more values to the datastore * Move S3 config to datastore * First pass the config -> db migrator * Add the start of the video config apis * It builds pointing everything away from the config * Tweak ffmpeg path error message * Backup database every hour. Closes #549 * Config + defaults + migration work for db * Cleanup logging * Remove all the old config structs * Add descriptive info about migration * Tweak ffmpeg validation logic * Fix db backup path. backup on db version migration * Set video and s3 configurations * Update api spec with new config endpoints * Add migrator for stats file * Commit updated API documentation * Use a dynamic system port for internal HLS writes. Closes #577 (#626) * Use a dynamic system port for internal HLS writes. Closes #577 * Cleanup * YP key migration to datastore * Create a backup directory if needed before migrations * Remove config test that no longer makes sense. Cleanup. * Change number types from float32 to float64 * Update automated test suite * Allow restoring a database backup via command line flags. Closes #549 * Add new hls segment config api * Commit updated API documentation * Update apis to require a value container property * add socialHandles api * Commit updated API documentation * Add new latancy level setting to replace segment settings * Commit updated API documentation * Fix spelling * Commit updated API documentation * hardcode a json api of available social platforms * Add additional icons * Return social handles in server config api * Add socialhandles validation to test * Move list of hard coded social platforms to an api * Remove audio only code from transcoder since we do not use it * Add latency levels api + snapshot of video settings as current broadcast * Add config/serverurl endpoint * Return 404 on YP api if disabled * Surface stream title in YP response * Add stream title to web ui * Cleanup log message. Closes #520 * Rename ffmpeg package to transcoder * Add ws package for testing * Reduce chat backlog to past 5hrs, max 50. Closes #548 * Fix error formatting * Add endpoint for resetting yp registration * Add yp/reset to api spec. return status in response * Return zero viewer count if stream is offline. Closes #422 * Post-rebase fixes * Fix merge conflict in openapi file * Commit updated API documentation * Standardize controller names * Support setting the stream key via the command line. Closes #665 * Return social handles with YP data. First half of https://github.com/owncast/owncast-yp/issues/28 * Give the YP package access to server status regardless if enabled or not * Change delay in automated tests * Add stream title integration API. For #638 * Commit updated API documentation * Add storage to the migrator * Missing returning NSFW value in server config * Add flag to ignore websocket client. Closes #537 * Add error for parsing broadcaster metadata * Add support for a cli specified http server port. Closes #674 * Add cpu usage levels and a temporary mapping between it and libx264 presets * Test for valid url endpoint when saving s3 config * Re-configure storage on every stream to allow changing storage providers * After 5 minutes of a stream being stopped clear the stream title * Hide viewer count once stream goes offline instead of when player stops * Pull steamTitle from the status that gets updated instead of the config * Commit updated API documentation * Optionally show stream title in the header * Reset stream title when server starts * Show chat action when stream title is updated * Allow system messages to come back in persistence * Split out getting chat history for moderation + fix tests * Remove server title and standardize on name only * Commit updated API documentation * Bump github.com/aws/aws-sdk-go from 1.37.1 to 1.37.2 (#680) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.1 to 1.37.2. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.1...v1.37.2) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add video variant and stream latency config file migrator * Remove mostly unused disable upgrade check bool * Commit updated API documentation * Allow bundling the admin from the 0.0.6 branch * Fix saving port numbers * Use name instead of old title on window focus * Work on latency levels. Fix test to use levels. Clean up transcoder to only reference levels * Another place where title -> name * Fix test * Bump github.com/aws/aws-sdk-go from 1.37.2 to 1.37.3 (#690) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.2 to 1.37.3. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.2...v1.37.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update dependabot config * Bump github.com/aws/aws-sdk-go from 1.37.3 to 1.37.5 (#693) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.3 to 1.37.5. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.3...v1.37.5) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump video.js from 7.10.2 to 7.11.4 in /build/javascript (#694) * Bump video.js from 7.10.2 to 7.11.4 in /build/javascript Bumps [video.js](https://github.com/videojs/video.js) from 7.10.2 to 7.11.4. - [Release notes](https://github.com/videojs/video.js/releases) - [Changelog](https://github.com/videojs/video.js/blob/main/CHANGELOG.md) - [Commits](https://github.com/videojs/video.js/compare/v7.10.2...v7.11.4) Signed-off-by: dependabot[bot] <support@github.com> * Commit updated Javascript packages Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Owncast <owncast@owncast.online> * Make the latency migrator dynamic so I can tweak values easier * Split out fetching ffmpeg path from validating the path so it can be changed in the admin * Some commenting and linter cleanup * Validate the path for a logo change and throw an error if it does not exist * Logo change requests have to be a real file now * Cleanup, making linter happy * Format javascript on push * Only format js in master * Tweak latency level values * Remove unused config file examples * Fix thumbnail generation after messing with the ffmpeg path getter * Reduce how often we report high hardware utilization warnings * Bundle the 0.0.6 branch version of the admin * Return validated ffmpeg path in admin server config * Change the logo to be stored in the data directory instead of webroot * Bump postcss from 8.2.4 to 8.2.5 in /build/javascript (#702) Bumps [postcss](https://github.com/postcss/postcss) from 8.2.4 to 8.2.5. - [Release notes](https://github.com/postcss/postcss/releases) - [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/postcss/postcss/compare/8.2.4...8.2.5) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Default config file no longer used * don't show stream title when offline addresses https://github.com/owncast/owncast/issues/677 * Remove auto-clearing stream title. #677 * webroot -> data when using logo as thumbnail * Do not list websocket/access token create/delete as integration APIs * Commit updated API documentation * Bundle updated admin * Remove pointing to the 0.0.6 admin branch * Linter cleanup * Linter cleanup * Add donations and follow links to show up under social handles * Prettified Code! * More linter cleanup * Update admin bundle * Remove use of platforms.js and return icons with social handles. Closes #732 * Update admin bundle * Support custom config path for use in migration * Remove unused platform-logos.gif * Reduce log level of message * Remove unused logo files in static dir * Handle dev vs. release build info * Restore logo.png for initial thumbnail * Cleanup some files from the build process that are not needed * Fix incorrect build-time injection var * Fix missing file getting copied to the build * Remove console directory message. * Update admin bundle * Fix comment * Report storage setup error * add some value set error checking * Use validated dynamic ffmpeg path for animated gif preview * Make chat message links be white so they don't hide in the bg. Closes #599 * Restore conditional that was accidentally removed Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com> Co-authored-by: Owncast <owncast@owncast.online> Co-authored-by: Ginger Wong <omqmail@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: nebunez <uoj2y7wak869@opayq.net> Co-authored-by: gabek <gabek@users.noreply.github.com>
234 lines
5.3 KiB
Go
234 lines
5.3 KiB
Go
package utils
|
|
|
|
import (
|
|
"bytes"
|
|
"errors"
|
|
"fmt"
|
|
"io/ioutil"
|
|
"net/url"
|
|
"os"
|
|
"os/exec"
|
|
"path"
|
|
"path/filepath"
|
|
"strings"
|
|
|
|
"github.com/mssola/user_agent"
|
|
log "github.com/sirupsen/logrus"
|
|
"github.com/yuin/goldmark"
|
|
"github.com/yuin/goldmark/extension"
|
|
"github.com/yuin/goldmark/renderer/html"
|
|
"mvdan.cc/xurls"
|
|
)
|
|
|
|
// GetTemporaryPipePath gets the temporary path for the streampipe.flv file.
|
|
func GetTemporaryPipePath() string {
|
|
return filepath.Join(os.TempDir(), "streampipe.flv")
|
|
}
|
|
|
|
// DoesFileExists checks if the file exists.
|
|
func DoesFileExists(name string) bool {
|
|
if _, err := os.Stat(name); err != nil {
|
|
if os.IsNotExist(err) {
|
|
return false
|
|
}
|
|
}
|
|
|
|
return true
|
|
}
|
|
|
|
// GetRelativePathFromAbsolutePath gets the relative path from the provided absolute path.
|
|
func GetRelativePathFromAbsolutePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
file := pathComponents[len(pathComponents)-1]
|
|
|
|
return filepath.Join(variant, file)
|
|
}
|
|
|
|
func GetIndexFromFilePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
|
|
return variant
|
|
}
|
|
|
|
// Copy copies the file to destination.
|
|
func Copy(source, destination string) error {
|
|
input, err := ioutil.ReadFile(source)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return ioutil.WriteFile(destination, input, 0600)
|
|
}
|
|
|
|
// Move moves the file to destination.
|
|
func Move(source, destination string) error {
|
|
return os.Rename(source, destination)
|
|
}
|
|
|
|
// IsUserAgentABot returns if a web client user-agent is seen as a bot.
|
|
func IsUserAgentABot(userAgent string) bool {
|
|
if userAgent == "" {
|
|
return false
|
|
}
|
|
|
|
botStrings := []string{
|
|
"mastodon",
|
|
"pleroma",
|
|
"applebot",
|
|
}
|
|
|
|
for _, botString := range botStrings {
|
|
if strings.Contains(strings.ToLower(userAgent), botString) {
|
|
return true
|
|
}
|
|
}
|
|
|
|
ua := user_agent.New(userAgent)
|
|
return ua.Bot()
|
|
}
|
|
|
|
func RenderSimpleMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
return buf.String()
|
|
}
|
|
|
|
func RenderPageContentMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.GFM,
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
return buf.String()
|
|
}
|
|
|
|
// GetCacheDurationSecondsForPath will return the number of seconds to cache an item.
|
|
func GetCacheDurationSecondsForPath(filePath string) int {
|
|
if path.Base(filePath) == "thumbnail.jpg" {
|
|
// Thumbnails re-generate during live
|
|
return 20
|
|
} else if path.Ext(filePath) == ".js" || path.Ext(filePath) == ".css" {
|
|
// Cache javascript & CSS
|
|
return 60
|
|
} else if path.Ext(filePath) == ".ts" {
|
|
// Cache video segments as long as you want. They can't change.
|
|
// This matters most for local hosting of segments for recordings
|
|
// and not for live or 3rd party storage.
|
|
return 31557600
|
|
} else if path.Ext(filePath) == ".m3u8" {
|
|
return 0
|
|
}
|
|
|
|
// Default cache length in seconds
|
|
return 30
|
|
}
|
|
|
|
func IsValidUrl(urlToTest string) bool {
|
|
_, err := url.ParseRequestURI(urlToTest)
|
|
if err != nil {
|
|
return false
|
|
}
|
|
|
|
u, err := url.Parse(urlToTest)
|
|
if err != nil || u.Scheme == "" || u.Host == "" {
|
|
return false
|
|
}
|
|
|
|
return true
|
|
}
|
|
|
|
// ValidatedFfmpegPath will take a proposed path to ffmpeg and return a validated path.
|
|
func ValidatedFfmpegPath(ffmpegPath string) string {
|
|
if ffmpegPath != "" {
|
|
if err := VerifyFFMpegPath(ffmpegPath); err == nil {
|
|
return ffmpegPath
|
|
} else {
|
|
log.Warnln(ffmpegPath, "is an invalid path to ffmpeg will try to use a copy in your path, if possible")
|
|
}
|
|
}
|
|
|
|
// First look to see if ffmpeg is in the current working directory
|
|
localCopy := "./ffmpeg"
|
|
hasLocalCopyError := VerifyFFMpegPath(localCopy)
|
|
if hasLocalCopyError == nil {
|
|
// No error, so all is good. Use the local copy.
|
|
return localCopy
|
|
}
|
|
|
|
cmd := exec.Command("which", "ffmpeg")
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
log.Fatalln("Unable to determine path to ffmpeg. Please specify it in the config file.")
|
|
}
|
|
|
|
path := strings.TrimSpace(string(out))
|
|
return path
|
|
}
|
|
|
|
// VerifyFFMpegPath verifies that the path exists, is a file, and is executable.
|
|
func VerifyFFMpegPath(path string) error {
|
|
stat, err := os.Stat(path)
|
|
|
|
if os.IsNotExist(err) {
|
|
return errors.New("ffmpeg path does not exist")
|
|
}
|
|
|
|
if err != nil {
|
|
return fmt.Errorf("error while verifying the ffmpeg path: %s", err.Error())
|
|
}
|
|
|
|
if stat.IsDir() {
|
|
return errors.New("ffmpeg path can not be a folder")
|
|
}
|
|
|
|
mode := stat.Mode()
|
|
//source: https://stackoverflow.com/a/60128480
|
|
if mode&0111 == 0 {
|
|
return errors.New("ffmpeg path is not executable")
|
|
}
|
|
|
|
return nil
|
|
}
|