owncast/core/chat/server.go
Gabe Kangas b6f68628c0
Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
2021-07-19 19:22:29 -07:00

318 lines
7.2 KiB
Go

package chat
import (
"encoding/json"
"net/http"
"sync"
"time"
log "github.com/sirupsen/logrus"
"github.com/gorilla/websocket"
"github.com/owncast/owncast/core/chat/events"
"github.com/owncast/owncast/core/data"
"github.com/owncast/owncast/core/user"
"github.com/owncast/owncast/core/webhooks"
"github.com/owncast/owncast/utils"
)
var _server *ChatServer
type ChatServer struct {
mu sync.RWMutex
seq uint
clients map[uint]*ChatClient
maxClientCount uint
// send outbound message payload to all clients
outbound chan []byte
// receive inbound message payload from all clients
inbound chan chatClientEvent
// unregister requests from clients.
unregister chan *ChatClient
}
func NewChat() *ChatServer {
server := &ChatServer{
clients: map[uint]*ChatClient{},
outbound: make(chan []byte),
inbound: make(chan chatClientEvent),
unregister: make(chan *ChatClient),
maxClientCount: handleMaxConnectionCount(),
}
return server
}
func (s *ChatServer) Run() {
for {
select {
case client := <-s.unregister:
if _, ok := s.clients[client.id]; ok {
s.mu.Lock()
delete(s.clients, client.id)
close(client.send)
s.mu.Unlock()
}
case message := <-s.inbound:
s.eventReceived(message)
}
}
}
// Addclient registers new connection as a User.
func (s *ChatServer) Addclient(conn *websocket.Conn, user *user.User, accessToken string, userAgent string) *ChatClient {
client := &ChatClient{
server: s,
conn: conn,
User: user,
ipAddress: conn.RemoteAddr().String(),
accessToken: accessToken,
send: make(chan []byte, 256),
UserAgent: userAgent,
ConnectedAt: time.Now(),
}
s.mu.Lock()
{
client.id = s.seq
s.clients[client.id] = client
s.seq++
}
s.mu.Unlock()
log.Traceln("Adding client", client.id, "total count:", len(s.clients))
go client.writePump()
go client.readPump()
client.sendConnectedClientInfo()
if getStatus().Online {
s.sendUserJoinedMessage(client)
s.sendWelcomeMessageToClient(client)
}
return client
}
func (s *ChatServer) sendUserJoinedMessage(c *ChatClient) {
userJoinedEvent := events.UserJoinedEvent{}
userJoinedEvent.SetDefaults()
userJoinedEvent.User = c.User
if err := s.Broadcast(userJoinedEvent.GetBroadcastPayload()); err != nil {
log.Errorln("error adding client to chat server", err)
}
// Send chat user joined webhook
webhooks.SendChatEventUserJoined(userJoinedEvent)
}
func (s *ChatServer) ClientClosed(c *ChatClient) {
s.mu.Lock()
defer s.mu.Unlock()
c.close()
if _, ok := s.clients[c.id]; ok {
log.Debugln("Deleting", c.id)
delete(s.clients, c.id)
}
}
func (s *ChatServer) HandleClientConnection(w http.ResponseWriter, r *http.Request) {
if data.GetChatDisabled() {
_, _ = w.Write([]byte(events.ChatDisabled))
return
}
// Limit concurrent chat connections
if uint(len(s.clients)) >= s.maxClientCount {
log.Warnln("rejecting incoming client connection as it exceeds the max client count of", s.maxClientCount)
_, _ = w.Write([]byte(events.ErrorMaxConnectionsExceeded))
return
}
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Debugln(err)
return
}
accessToken := r.URL.Query().Get("accessToken")
if accessToken == "" {
log.Errorln("Access token is required")
// Return HTTP status code
conn.Close()
return
}
// A user is required to use the websocket
user := user.GetUserByToken(accessToken)
if user == nil {
_ = conn.WriteJSON(events.EventPayload{
"type": events.ErrorNeedsRegistration,
})
// Send error that registration is required
conn.Close()
return
}
// User is disabled therefore we should disconnect.
if user.DisabledAt != nil {
log.Traceln("Disabled user", user.Id, user.DisplayName, "rejected")
_ = conn.WriteJSON(events.EventPayload{
"type": events.ErrorUserDisabled,
})
conn.Close()
return
}
userAgent := r.UserAgent()
s.Addclient(conn, user, accessToken, userAgent)
}
// Broadcast sends message to all connected clients.
func (s *ChatServer) Broadcast(payload events.EventPayload) error {
data, err := json.Marshal(payload)
if err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
for _, client := range s.clients {
if client == nil {
continue
}
select {
case client.send <- data:
default:
close(client.send)
delete(s.clients, client.id)
}
}
return nil
}
func (s *ChatServer) Send(payload events.EventPayload, client *ChatClient) {
data, err := json.Marshal(payload)
if err != nil {
log.Errorln(err)
return
}
client.send <- data
}
// DisconnectUser will forcefully disconnect all clients belonging to a user by ID.
func (s *ChatServer) DisconnectUser(userID string) {
s.mu.Lock()
clients, err := GetClientsForUser(userID)
s.mu.Unlock()
if err != nil || clients == nil || len(clients) == 0 {
log.Debugln("Requested to disconnect user", userID, err)
return
}
for _, client := range clients {
log.Traceln("Disconnecting client", client.User.Id, "owned by", client.User.DisplayName)
go func(client *ChatClient) {
event := events.UserDisabledEvent{}
event.SetDefaults()
// Send this disabled event specifically to this single connected client
// to let them know they've been banned.
_server.Send(event.GetBroadcastPayload(), client)
// Give the socket time to send out the above message.
// Unfortunately I don't know of any way to get a real callback to know when
// the message was successfully sent, so give it a couple seconds.
time.Sleep(2 * time.Second)
// Forcefully disconnect if still valid.
if client != nil {
client.close()
}
}(client)
}
}
func (s *ChatServer) eventReceived(event chatClientEvent) {
var typecheck map[string]interface{}
if err := json.Unmarshal(event.data, &typecheck); err != nil {
log.Debugln(err)
}
eventType := typecheck["type"]
switch eventType {
case events.MessageSent:
s.userMessageSent(event)
case events.UserNameChanged:
s.userNameChanged(event)
default:
log.Debugln(eventType, "event not found:", typecheck)
}
}
func (s *ChatServer) sendWelcomeMessageToClient(c *ChatClient) {
// Add an artificial delay so people notice this message come in.
time.Sleep(7 * time.Second)
welcomeMessage := utils.RenderSimpleMarkdown(data.GetServerWelcomeMessage())
if welcomeMessage != "" {
s.sendSystemMessageToClient(c, welcomeMessage)
}
}
func (s *ChatServer) sendAllWelcomeMessage() {
welcomeMessage := utils.RenderSimpleMarkdown(data.GetServerWelcomeMessage())
if welcomeMessage != "" {
clientMessage := events.SystemMessageEvent{
Event: events.Event{},
MessageEvent: events.MessageEvent{
Body: welcomeMessage,
},
}
clientMessage.SetDefaults()
_ = s.Broadcast(clientMessage.GetBroadcastPayload())
}
}
func (s *ChatServer) sendSystemMessageToClient(c *ChatClient, message string) {
clientMessage := events.SystemMessageEvent{
Event: events.Event{},
MessageEvent: events.MessageEvent{
Body: message,
},
}
clientMessage.SetDefaults()
s.Send(clientMessage.GetBroadcastPayload(), c)
}
func (s *ChatServer) sendActionToClient(c *ChatClient, message string) {
clientMessage := events.ActionEvent{
MessageEvent: events.MessageEvent{
Body: message,
},
}
clientMessage.SetDefaults()
clientMessage.RenderBody()
s.Send(clientMessage.GetBroadcastPayload(), c)
}