Académique Documents
Professionnel Documents
Culture Documents
------------------------
- Fixed: Handling for unknown sid (0 or a dummy sid listed in dummy-services) was
broken in 0.9.0.
- Fixed: Incoming requests for dummy sids should not retain the dummy when
forwarded, now changed to 0.
- Fixed: Per profile max-cw-wait could not be set below 1s (now both per profile
and global allow down to 100 ms).
NOTE: While it may now be possible to configure such low values, whether it works
or not below 1s is still undefined.
- Fixed: Probing was sometimes attempted even when there was only one candidate
connector.
- Fixed: Radegast support updated to make sense with 0.9.x.
- Fixed: Emm handling for extended newcamd in systems with signifcant provider-
idents.
- Fixed: Now possible to move messages between profiles from plugins, changes to
networkid/caid were previously ignored.
- Fixed: Per user sid lists provided by custom usermanagers were ignored for multi-
context sessions (Csp, ExtNewcamd).
- Fixed: The default-deny-list wasn't checked for CspConnect connection attempts.
- Fixed: Closing of old sessions (on new login) wasn't working for ExtNewcamd.
- Fixed: Debug mode for the "*" pseudo profile (Csp, ExtNewcamd) couldn't be turned
off via config (attribute ignored).
- Fixed: Enigma1/2 bouquet file generator in status web now works for multi-profile
setups. Additionally, it will
include services for which no mapping data exist (unscrambled, or never watched).
Still requires enigma service files.
- Fixed: Using manual service mapping (can-decode list) for one connector would
affect discovery of the listed services
for other connectors in the same profile.
- Fixed: It was not possible to specify manual service mapping lists for csp and
chameleon connectors, for more than
one profile at a time.
- Fixed: Removing a profile from a user that had used csp-connect had no effect, as
it remained locally cached at the
client proxy, and no mechanism existed for removing it or blocking the now
disallowed requests.
- Fixed: Missing log warning and cws event for csp-connector login failure (on bad
credentials).
- Fixed: Massive memory leaks when using high volumes of ClusteredCache udp updates
in broken/assymetric clusters, or
when using third party relayers (probably still a bad idea to do this however).
- Changed: Two ecms (or dcw replies) with the same payload data but different dvb
table ids (even/0x80 vs odd/0x81) are
now considered identical by the proxy. Implications unknown.
- Changed: ClusteredCache transport format now includes meta-data such as ca-id and
network-id.
NOTE: This breaks compatibility with previous versions. If one node has the new
format, all nodes must be upgraded.
The source ip's of any received incompatible updates are listed in the property
'version-mismatch' (when possible).
- Changed: Enabling debug for a user now also enables web transaction logs for that
user (same as profile debug).
NOTE: Kicking the users sessions may be required before this takes effect (both
after enabling and disabling).
- Added: Experimental resend/broadcast mechanism for ClusteredCache, to make cache-
only nodes slightly more feasible.
This allows a node to ask all peers for a cw that doesn't exist locally and is
taking more than max-cache-wait / 2.
- Added: Option to automatically add any peers that are sending updates to
ClusteredCache.
- Added: Cws event for found service now shown in the status-web (despite the
command being called error-log).
- Added: Experimental sid cache linking feature, allows cache hits based on meta-
data like sid rather than ecm payload.
Links added via the web commands are stored in etc/links.cfg (this file is
monitored for changes and autoloaded also).
- Added: More sanity checks to DcwFilterPlugin (checksums, length and an option to
monitor all replies for duplicates).
- Added: DreamboxPlugin (csp-agent) support for more/older dm500 images. Basic
(unsecured) file upload support.
- Added: proxy-reference.html updated with highlighting for important elements and
additional examples.
- Added: CacheCoveragePlugin, for visualizing the contents and show coverage per
transponder/mux.
- Added: New AdvXmlUserManager to show how to add um-specific functionality without
changing the core or existing um's.
- Added: Some basic sanity checks for incoming ClusteredCache udp updates (DCW
checksum, zero counting). Default enabled.
- Changes to proxy.xml:
Added: Element <auto-add-peers> to ClusteredCache <cache-config> (true/false,
default: false). Add peers automatically
if they send you traffic (adding them as targets for outgoing packets).
Added: Element <cw-validation> to ClusteredCache <cache-config>. Attributes
'checksum', 'zero-counting', 'log-warnings'
(true/false, default true). Determines whether to drop remote-origin cw's with
bad checksum, and more than 4 zeroes
(but less than 8, so intentionally blanked out cw's will not be affected by this
check).
Added: Attribute 'enable-service-linking' to <cache-handler> (true/false,
default: false). Enables sid cache linker.
Added: Attribute 'allow-different-ip' to <user-manager>, allows newcamd
connections from different ip sources
for the same user.
Added: Element <hide-radio-services> to <mapper> (true/false, default: true).
Show/hide radio services in web/xml api.
- Added: When using only asynchronous connectors, it is now possible for a client
session to get a cache hit even after
a forward to card was initiated. This can result in transactions with both F and
C/R flags.
- Added: Last-seen data now also contains entries for failed login/connect
attempts, available through a new status
command 'login-failures' (available to all users, but non-admins can only see
attempts made with their user name).
- Added: New interface ReplyFilter that plugins can use to intercept and
alter/block DCW's as they're returned from
connectors, before they're processed by the proxy (possibly find and delete bad
CW's). See README.Plugins.txt.
A DcwFilterPlugin that illustrates this and blocks some common bad responses is
included.
- Added: Fixes for running under jamvm on embedded systems (including the broken
auto-generating of the config template).
- Added: New status commands for troubleshooting: 'export-services', 'system-
properties', 'system-threads', 'file-log'
and 'proxy-plugins'.
- Added: Option to configure the date-format used by the default logger (allows
easier fail2ban monitoring).
- Added: Arbitrary meta-data/remote info can now be returned by connectors (for
display/troubleshooting/statistics).
- Added: Multiple client ids (oscam, scam, rq-sssp etc).
- Added: More example plugins included (and updated versions of the rest).
- Added: JVM version check. The proxy will refuse to start with anything but a sun
jvm. If you're absolutely sure, this
can be overridden by adding the following to the java cmd line:
-Dcom.bowman.cardserv.allowanyjvm=true
- Changes to proxy.xml:
Added: Element <csp-connect> to <status-web> (to receive csp-connections, enabled
by default).
Added: Element <csp-connector> to <connectors> (to define csp-connectors). See
proxy-reference.html.
Added: Element <chameleon-connector> to <connectors> (connector to a
newcs/chameleon setup as mgcamd). Same as a
newcamd-connector, except it is not bound to a profile, always asynchronous and
ignoring the client-id setting.
Added: Element <extended-newcamd> to <profiles> (unbound port for extended
newcamd protocol, as used between mgcamd
and newcs). Allows mgcamd to use multiple systems (all profiles the user has
access to) over a single connection.
Added: Element <log-dateformat> to <logging> (optional java SimpleDateFormat
string to use for the standard logs).
Added: Attribute 'provider-idents' to <profile> (optional, allows listing of
provider-idents, even with no connectors).
Added: Attribute 'require-provider-match' to <profile> (true/false, default:
true). Set to false if you know that for
this profile, provider idents in ecm requests do not need to match those on the
cards (this is the case for irdeto).
NOTE: If require-provider-match is false, provider-idents will get 000000 added
automatically. Conversely, if only
ident 000000 is specified for a profile, require-provider-match defaults to false
instead of true when omitted.
Added: Attribute 'provider-idents' to <newcamd-connector>, (optional, overrides
the idents from the server/card).
Added: Attribute 'exclusive' to <can-decode-services> (true/false, default:
false). Set to true for a list to indicate
that there should be no probing done for the connector, only those services
listed are to be considered decodable.
Added: Attribute 'profile' to <can-decode-services> and <cannot-decode-services>.
Only applicable for connector types
csp-connector and chameleon-connector, where multiple lists can be used to
specify services for several profiles.
that there should be no probing done for the connector, only those services
listed are to be considered decodable.
Changed: Attribute 'provider' for <service-file> changed to 'filter' to avoid
confusion. If provider-idents have been
specified correctly for the profile, there is no longer any need to repeat that
list in the case of cccam parsing.
Changed: All elements that allowed hex sid lists to be specified (per connector
or profile) now accept an alternate
syntax sid:cid (where cid is custom id, used for situations like the irdeto chid
where sid alone is not enough).
NOTE: <allow-services> is an exception, checks against that list are made with
sid only.
Changed: Element <unknown-sid> removed from <mapper>, replaced with <dummy-
services> that may contain multiple sids.
Use this to list any fake sids used by limited clients that can't know the real
one, to avoid interference with maps.
Added: Element <redundant-forwarding> to <mapper> (true/false, default: false).
Can be set globally or per profile,
as with other mapping settings. Transactions that trigger redundant forwarding
will get the new flag '2'.
Added: Attribute 'include-file-events' to <warning-threshold> (true/false,
default: true). Setting this to false
disables the 'file-log' http query (no file log events will be intercepted for
display on the web).
- Added: New attribute per profile 'network-id' (the original dvb network id, 4 hex
digits). This will uniquely
identify profiles between proxies, instead of ca-id. It will also be used in
filtering enigma services files.
See proxy-reference.html for info about how to find the id if you don't know your
dvb-network.
- Added: Optional mapping table to auto-assign newcamd connectors to profiles based
on ca-id, to make this work as
before even when multiple profiles use the same ca-id (no longer any need to set
0000, use the correct id always).
Use this only when it is unknown which card a remote newcamd server contains (or
when this changes randomly).
- Added: ClusteredCache now avoids locking for requests where a remotely received
cache lock has the same ip as
requesting client, to deal with situations where both connectors and cache
sharing are used to link multiple proxies
together (prevents incoming ecm requests from a remote proxy being blocked by the
cache lock from that same proxy).
- Added: Plugins can now affect connector selection, if they implement the
interface CwsSelector. They'll be given a
chance to exclude connectors from the selection process for each message.
- Added: Cache hits where the cached request had a different ca-id will now be
blocked, to avoid misleading clients
that are sending requests to the wrong profile. This can be disabled using new
global setting <block-caid-mismatch>.
- Added: Disconnected connectors now show when they went offline.
- Added: Newcamd OSD messages are now sent to Acamd as well (previously only to
Mgcamd).
- Added: More user events in the remote api (login, logout, login failure). Plugins
can use these as well.
- Changed: ClusteredCache now uses a much more compact format to reduce bandwidth
(not compatible with old versions).
- Fixed: ClusteredCache bandwidth usage estimates are now properly calculated even
with multiple peers and sync-period.
- Fixed: Minor browser-specific issues with the status web layouts (clear the
browser cache or force a reload).
- Changes to proxy.xml:
Added: Attribute 'network-id' to <profile> (4 hex digits, i.e "a027"). Set this
to the original dvb network id.
Added: Element <block-caid-mismatch> (true/false, default: true) to section ca-
profiles.
Added: Element <caid-profile-map> to <newcamd-connector> (caid=profilename,
entries separated by space). This element
is mandatory if no profile attribute is set for the connector.
For example: <caid-profile-map>0b00=cable 0500=terrestrial</caid-profile-map)
0.8.12 - 2009-06-29
- Fixed: No forwarding attempt will now be made if a cache timeout meant that max-
cw-wait was already exceeded.
- Added: Experimental broadcasting of ecms without sid to all non-congested
connectors in profile (off by default).
- Added: Client id for DiabloCam wifi (uw?).
- Added: Command to clean the last-seen log.
- Added: Made the connector reconnect attempts more visible and consistent in the
status web.
- Changed: Radegast sessions are no longer added to disconnected users/last-seen
log.
- Changed: Connectors that fail on proxy startup are now added anyway, to make them
visible via the http/xml api.
NOTE: If the failed connector has no profile specified, it will still not be
visible until it has connected properly.
- Fixed: Error when unloading the LoggingPlugin.
- Fixed: Lost service warnings are no longer shown for sids that have been made
sticky using can-decode-services.
- Fixed: Slow memory leak in ClusteredCache when using sync-period.
- Fixed: Typos in the generated proxy.xml template.
- Fixed: Inconsistent username case handling.
- Fixed: Card-data with extra trailing bytes would break sessions for non-au users
(rqcs).
- Fixed: The open-access handling should now work for xmlusermanager as well as
simpleusermanager.
- Updated ConaxConnector to latest version.
- Changes to proxy.xml:
Added: Element <hide-disabled-connectors> (true/false, default: false) to section
mapper.
Added: Element <broadcast-missing-sid> (true/false, default: false) to section
mapper.
0.8.11 - 2009-05-01
- Added: Jvm stats in the debug logging: heap used/allocated, thread count, file
descriptors (used and max, unix only).
- Added: UserManager interface now has a getDisplayName() method, to allow for an
optional display-name attribute.
- Added: Client id for Rq-echo-client and sbcl.
- Added: Config and Admin sections (status-web) are now disabled by default. List
users that should have access to
these in the new super-users element in the status-web section.
- Added: MessagingPlugin can now send automated email with the same type of
triggers as for mgcamd-osd.
- Added: Msg filtering now shows up in LoggingPlugin (assuming the filtering plugin
used msg.setFilteredBy() to do it).
- Added: SimpleUserManager now allows for open access, accepting any newcamd
connections as long as they have the right
(common) password. Open access can be restricted to specific profiles or to
usernames starting with a certain prefix.
NOTE: Random user names will be assigned, but the display-name will be whatever
the client specified.
- Added: Made it possible to configure card-data (non-au) for newcamd ports without
editing the card-files manually.
- Added: A parser for using CCcam.channelinfo as the services-file for a profile.
As with enigma files you need to
specify which sids are relevant for this profile by creating a filter. See proxy-
reference.html for details.
- Added: Externally loaded plugins can now be individually re-loaded just by
replacing the jar file.
- Fixed: When using the status-web without ssl, any specified bind-ip was ignored.
- Fixed: Distribution tars finally corrected.
- Fixed: Enigma bouquet-file generator in the status web might actually produce
valid files now (enigma services-file).
- Fixed: Status-web session transfer from script/ajax context to browser context
via cookie was always broken.
- Fixed: Made card-data parsing less sensetive to errors/truncated data.
- Changed: More minor tweaks of the status web views.
- Changed: Blowfish encryption tool trtest.jar renamed to: fishenc.jar
- Updated the codemirror xml/js editor (used in the status-web) to version 0.61.
- Changes to proxy.xml:
Added: Element <super-users> to <status-web>. List of user names that should have
access to the Config and Admin
sections (the users must have admin="true" or they'll be ignored).
Added: Element <open-access> to <auth-config> (SimpleUserManager/XmlUserManager).
See proxy-reference.html.
Added: Attribute 'ca-id' to <card-data> (newcamd listen ports). The ca-id
returned to clients, when type="config".
Added: Element <providers> to <card-data>. List of providers returned to clients
when type="config". 3 bytes each,
separated by comma, i.e: 00 00 00,00 00 01,00 00 02
0.8.10 - 2009-03-08
- Fixed: The experimental strict synchronization for ClusteredCache now does what
it claimed to (sync-period).
- Fixed: Setting debug="true" for the logging now outputs any stacktraces at
WARNING level, as indicated in the docs.
- Fixed: Sessions weren't always being properly disconnected when session timeout
was hit (introduced in 0.8.9).
- Fixed: Newcamd sequence nr wasn't set correctly in EMM replies (caused some
clients to log errors).
- Fixed: Possible socket handle leak when new sessions couldn't be created. This
may have caused listen ports to close.
- Fixed: The default value for user attribute max-connections is now calculated
correctly, based on the total number of
newcamd listen-ports in the profiles the user has access to. See INFO logging on
startup and on connect for the values.
- Fixed: If a user exceeds the max-connections count, the session that is closed is
now always the oldest one.
- Changed: Slightly improved views and more stat values in the example status web.
- Changed: ClusteredCache now accepts multiple proxies running on the same host
name (on different ports).
- Changed: ClusteredCache is now slightly less inefficient with its bandwidth usage
(still much room for improvement).
- Added: ClusteredCache now pings each peer regularly to keep track of latency
(breaks compatibility with old versions).
- Added: ClusteredCache additional stat counters (toggle debug for the cache to see
all).
- Added: Custom usermanager and cache implementations can now be loaded with the
plugin classloader (from a jar-file).
- Added: Additional custom connector implementations can now be loaded, also via
the plugin classloader if desired.
A new example connector is included with the plugis: ConaxConnector - It reads
local conax cards directly using java6
and pcsc card-readers. See README.ConaxConnector.txt for more info.
- Changes to proxy.xml:
Added: Attribute 'jar-file' to elements <user-manager> and <cache-handler>
(optional). If this is specified, the class
will be loaded by the plugin classloader from the named jar-file (path relative
to the plugins dir).
NOTE: This doesn't mean the implementations can be re-loaded dynamically,
restart is still required for new jars.
Added: It is now possible to specify custom connector implementations using the
same notation as the built in newcamd
and radegast connectors, but with the added attributes of 'class' and
optionally also 'jar-file'.
If a jar-file is specified the plugin classloader is used, as above. For
example:
<conax-connector name="lcard" profile="sat2"
class="com.bowman.cardserv.cws.ConaxCwsConnector" jar-file="conaxconnector.jar">
<!-- config goes here -->
</conax-connector>
0.8.9 - 2008-11-04
- Fixed: Possible infinite loop state for the file-change-watcher (would fill
sysout log once triggered).
- Added: The reset-connector ctrl command can now be used to delete all mappings
(full reset).
- Added: Client ids for cardlink.nl and octagon stbs.
- Added: A test-delay feature for the LoggingPlugin. This allows for a manually
added delay applied to all logged
requests (before they are processed). The delay can be set via status web
command, and is intended as tool for
finding the exact freeze-time for a given ca-profile, i.e: gradually increase the
delay until freeze and note the
full ecm transaction time (roundtrip) in the client logs. The LoggingPlugin has
also received a feature for sending
arbitrary newcamd-messages. This can be used to explore the capabilities of
different clients in realtime.
- Added: New setting for allow-services lists, per profile (inverse of block-
services).
- Added: The duration column in the status-web now shows the time since last zap
(if any has occured since connect).
- Added: Feature for sending keep-alives to clients as well as servers. Normally
only the client sends these in newcamd,
but most clients seem to ignore incoming keep-alives. It can be used to find dead
sessions faster in very large shares.
- Changed: XmlUserManager now considers deleted users as disabled (causing any
active sessions to be kicked).
- Changed: max-cw-wait can now be set per ca-profile, as it is typically different
from one ca-system to the next.
This also means all capacity estimates (status-web) can be made more accurate.
Use the test-delay feature of the
LoggingPlugin to find the freeze-time for each profile, and set the max-cw-wait
to this (or 1 second above).
- Changed: Moved more previously hard-coded settings to config. No need to touch
these unless you know what its about.
- Updated the codemirror xml/js editor (used in the status-web) to version 0.58.
- Misc minor fixes.
- Changes to proxy.xml:
Added: Element <max-threads> to <ca-profiles> (default: 1000). If this is reached
the proxy will stop accepting
connections until it drops again. The fixed default in 0.8.8 was 500.
Added: Element <session-timeout> to <ca-profiles> (default: 120, in minutes).
Maximum idle time for user sessions.
Added: Element <newcamd-maxmsgsize> to <ca-profiles> (default: 400). This is
CWS_NETMSGSIZE. The old default was 240,
so try that if you run into any problems related to message size.
Added: Element <allow-services> to section mapper. List of sids (hex). The
inverse of <block-services>, only sids
listed here will be passed through to connectors. This can help filter out
requests with bad ecms, and reduce
unecessary probing of the cards. As for block-services, it only makes sense to
specify this element per profile.
Added: Elements <max-cw-wait> and <congestion-limit> to <profile>. Same settings
as for the connection-manager, but
allows overriding the global setting per profile.
Added: Element <session-keepalive> to <ca-profiles> (default: 0, in minutes =
off). Sends keep-alives to clients.
Attributes are: exclude-clients (list of client names, default: "" = send to
all). Exclude listed client types.
- Changes to proxy.xml:
Added: Element <default-max-queue> to <connection-manager> (default: 50). Max
queue length allowed to build up on
one connector before the proxy assumes something has crashed and disconnects
it.
Added: Element <default-min-delay> to <connection-manager> (default: 10, in ms).
Delay inserted between consecutive
ecms to one connector in async mode. Workaround for servers that misbehave when
requests are too close together.
Added: Elements <max-queue> and <min-delay> to <newcamd-connector>/<radegast-
connector>, same as above but specified
per individual connector (allowing different values for different connectors).
Added: Attribute 'format' to <services-file> (default: enigma). Example for the
"simple" format:
<services-file format="simple">etc/services.properties</services-file> <!--
format e.g: 03fb=Service Name -->
0.8.7 - 2008-08-23
- Added: Connector info now contains details about the received newcamd card-data
(status web/xml api).
- Added: Client id for rqcamd.
- Fixed: Capacity estimates remained for cards that were disconnected, they are now
excluded from the totals.
- Fixed: Plugins only worked if all of them were externally loaded (from separate
jar file). I.e: LoggingPlugin failed.
- Fixed: A few unlikely memory leaks related to connectors ending up in a zombie
state (seemingly ok but locked).
- Changed: Only client id 0000 is now displayed as Generic, other unknowns will be
shown by the actual numerical id.
- Changed: Transaction flags are now in the order in which they are set internally
by csp, not alphabetical.
- Minor updates to the example plugins.
0.8.6 - 2008-07-27
- Changed: Xml status commands are now handled like the control commands, they can
be registered and added on the fly
by any user component. This also makes it possible to override the default
command handlers with your own.
- Changed: Reworked the "filters" interface to a more generic plugin framework, to
enable quick extensions that
are not directly related to the ecm traffic (LoggingFilter changed to
LoggingPlugin).
- Added: GeoipPlugin. Illustrates the plugin framework and command overriding (adds
a google maps + geoip mashup).
- Added: EmmAnalyzerPlugin. Gathers statistics regarding emms received from
clients.
- Added: MessagingPlugin. Auto mgcamd-osd replies for client session (e.g "service
unavailable") and mail to users.
- Added: New flag '1' (one), assigned to the first transaction a session performs
(instead of Z as before).
- Added: Stat counter for denied/blocked ecm's (flag N).
- Added: Missing client ids (cccam, evocamd, alexcs etc).
- Changes to proxy.xml:
Changed: Elements <proxy-filters>, <filter> and <filter-config> renamed to
<proxy-plugins>, <plugin> and
<plugin-config>.
Added: Attribute 'jar-file' to <proxy-plugin>. Allows loading the plugin using a
separate classloader, from an
external jar file in the "plugins" dir. This way plugins can be
replaced/reloaded at runtime without restarting
the proxy (they are reloaded and restarted each time the proxy.xml config is
touched/updated).
Added: Element <delay-missing-sid> to <connection-manager> (default: 100, in ms).
This adds a short delay for any
incoming request without sid, before the cache is checked. The idea is to
increase the likelyhood of a another
request for the same ecm (but with sid specified) arriving first in the cache.
This is to avoid having a large
number of clients waiting in the cache for a forward that might get routed to
the wrong card (because it had no sid).
0.8.5 - 2008-06-29
- Fixed: Proxy now sets correct service id in cache hit replies (i.e same as in the
request, unless 0).
- Fixed: Repeated web events for "cws connected" retries when the server returned
invalid card data (caid 0 or mismatch).
- Fixed: Connector name was sometimes missing in the transaction time breakdown
even though the F-flag was set.
- Fixed: XmlUserManager will now keep its last known working set of users when
proxy configuration changes are made.
- If a reply was received from a remote cache, the connector name in the
transaction data is now prefixed with:
"remote:" - to make it possible to distinguish when the same names are present
locally.
- "Last seen" session data now includes last known ip address for the user.
- Cache flag 'O' is now strictly for timeouts in the cache (max-cache-wait
exceeded). Added new flag 'Q' for aborts due
to forward failures (remote or local).
- Added filtering per profile to status web (only shown for users with access to
more than one, in events + channels).
- Added ecm load estimates to status web totals and ca-profiles (sums based on the
cws-connector data below).
- Added a tool for tracing configuration file use in the proxy. Set
-Dcom.bowman.cardserv.util.tracexmlcfg=true when
starting the proxy and all config access will be traced. This trace can be
written to file (etc/xmlcfg.txt) by using
the CtrlCommand "dump" from the admin page of the status web (or by using the
http/xml api directly).
- Changes to proxy.xml:
Added: Element <hide-names> for ClusteredCache (true/false, default: false). Set
to true to stop the cache from
sending the connector names to the remote targets (only makes sense if they are
untrusted/unknown and the names
contain sensetive information).
Changed: Attribute 'debug' for <profile> now defaults to true. The flag is only
used to enable the transaction
backlog and there is typically no reason not to have that.
0.8.4 - 2008-05-27
- Fixed: Old ecm replies being processed as card data on rapid newcamd reconnects
(caused parse errors and bogus ca-id
for connectors, leading to them being disabled).
- Fixed: Newcamd clients sending multiple async requests to the proxy would get
incorrect/duplicate sequence numbers
in their replies (effectively making the proxy incompatible with async mode).
- Fixed: Events for connectors with no profile configured (i.e those with auto-
detect) were hidden even from admin in
the web error-log.
- Fixed: Connection failures occuring during startup are no longer filtered in the
web view. Also replaced the firefox
alert error for when the web script can't reach the proxy to something less
cryptic.
- Connector re-connect behaviour on login failure improved (no longer logs
disconnects which would cause 3 sec loop).
- Flag T is now only used for actual timeouts when forwarding, transactions
affected by aborts/disconnects will receive
flag A instead.
- Service names will now include a [R] prefix for radio and [HD] for hdtv (based on
the type from the services file).
- Added more details to the LoggingFilter output (sequence numbers, sids, sessions)
to improve client troubleshooting.
- Duplicate newcamd messages (with the same sequence number) are now logged as
warnings for the CWS communication.
Also, if debug logging is set the same is done for client communication (although
these can occur normally when
zapping or due to local network lag/congestion and don't necessarily mean
trouble).
NOTE: If a server returns the same sequence id twice (without having been sent
that) it indicates something went
wrong on the server side, either some kind of overload situation or outright
bugs.
If it repeats it needs to be investigated further and resolved.
The proxy will now also check to make sure the sid in the reply matches the one
in the request, to help identify
error replies.
- For case-insensetive user managers (e.g SimpleUserManager), the stored case of
the user name is now used instead of
the one supplied by the client for the login.
- Changes to proxy.xml:
Added: Element <log-sid-mismatch> to <connection-manager> (true/false, default:
true). Allows turning off logging
warnings when sid in the server reply doesn't match what was in the request.
0.8.3 - 2008-04-15
- Fixed: CtrlCommands caused NPE if status-web was disabled or had failed to start.
- Fixed: Logging in case newcamd card data could not be parsed (WARNING level + the
offending data now logged).
- Fixed: Services from different profiles could be merged in the watched-services
xml reply if they had the same name.
- Fixed: Minor pending ecm leak in ClusteredCache.
- Changed asynchronous newcamd mode to be off by default as it caused problems with
some servers (you now have to
explicitly set <asynchronous>true</asynchronous> for each connector if you want
to use it).
- Further tweaked utilization estimates, now using different methods depending on
async/sync mode.
- Made it possible to manually override the service maps for each connector, by
specifying sid lists. Useful for
situations where the automatic service discovery is unreliable. Services that
aren't manually specified will still
be probed for automatically.
- Status web now shows services with full information (sid and profile), space and
context permitting.
Additionally, services listed per connector will be highlighted blue if a forward
occured for that service in the
last max-cw-age seconds (allows you to see roughly which services the ecm-load
value refers to).
- Changes to proxy.xml:
Added: Element <hard-congestion-limit> (true/false, default: true) to
<connection-manager>. See above.
Changed: Element <asynchronous> for <newcamd-connector> default value changed to
false.
Added: Elements <can-decode-services> and <cannot-decode-services> to all
connector types. Optional lists of
sids (hex) allowing manual overriding of the automatic service mapping. Note
that services already known to decode
on a connector will not be affected by the cannot-decode-services list (until a
manual reset is performed, or the
corresponding .dat file is deleted from the cache dir).
0.8.2 - 2008-03-07
- Congestion warnings are now logged only when no alternative connectors exist.
Also, the utilization estimate is now a
factor in determining congestion (i.e > 100% over the last 60 secs = congested
even when there are 0 pending requests).
- Made it possible to configure the threshold for logging CWS timeout events
(default 1, was 2 before 0.8.0).
The number of timeouts to allow in a row before disconnecting can also be set
(default 2, was 3 before 0.8.0).
- The max-connections value (per user) now defaults to the number of profiles the
user has access to, or the total
number of active profiles if there are no restrictions. Previous default was 1,
if you want to keep that limit you
will now have to explicitly set every user to max-connections="1" in the user
manager.
- Flag T now means only this: Timeout when forwarding (no response from CWS within
time limit, i.e max-cw-wait).
- New flag S introduced: Timeout in send queue (when trying to forward to
connector, should normally not occur).
- The log event (level FINE) that occurs when client sessions end now contains a
summary of the session state, to help
show why the client may have disconnected. If the user has debug="true" this is
logged with level INFO.
- If the ca-id for card data (received during login for a newcamd connector) is
0000, the proxy will fail the login and
try again later instead of disabling the connector. This may help with re-init
card issues that cause servers to
temporarily return empty card data.
If override-checks is true then this check is also skipped and the 0000 data
accepted as valid.
0.8.1 - 2008-02-28
0.8.0 - 2008-02-25
- More changes to support other jvms (removed references to sun base64, and httpd
now tries multiple ssl providers).
- Added a javascript xml-editor to the status-web, for quick config updates (based
on codemirror, source included).
- Forced parsing of proxy.xml to always use UTF-8 regardless of system locale (when
installed from both file and web).
- Cleaned up web backend and made it possible to extend it from user code (see
ClusteredCache, XmlUserManager source).
- Cleaned up the client side scripting for the status-web, it now uses xslt to
generate the markup (see xslt dir in war).
- All user-log transactions will now contain cws-name (if it was a transaction that
involved a forward).
- Added a new user-warning-log with potential problems from all user recent
transactions (40 most recent).
- Transactions marked as warnings will now contain additional debug information,
and show the time spent on each stage:
in cache, cws send queue, cws reply wait, client write back.
- Tweaked probing to avoid multiple probes for the same service and connector, and
avoid problems under high load.
- Fixed: Now possible to add new ca-profiles without restarting.
- Fixed: Transaction tracking now correctly deals with overlapping/asynchronous
newcamd traffic. Flag E is now always
included if the client reply was empty, regardless of the cause.
- ClusteredCache will now indicate in the log which remote proxy has the wrong
version (SEVERE event on startup).
Additionally the cache stats on the web will contain a version-mismatch property
with the same IP.
- Status web section "Sessions" (previously Users) will now contain last-seen
information for disconnected users and
an option to show the idle sessions. Note: No last-seen data will be shown if all
known users are connected.
This section is now shown for regular users (non-admin) but will only list
information related to their own sessions.
- Changes to proxy.xml:
Added: Element <warning-threshold> to <logging>. Defines which transactions
should be considered potential problems.
Attributes are: bad-flags (string list of all that should qualify), max-delay
(in ms).
This setting determines what will show in the user-warning-log for profiles
that have debug="true".
Note: changing it will not affect already recorded events.
Added: Element <external-connector-config> to <connector-manager>. Specifies an
external source for connector
definitions with the following elements: connector-file-url, connector-file-
key, update-interval (minutes).
Added: Element <asynchronous> to <newcamd-connector> (true/false, default: true).
A way to disable asynchronous mode.
0.7.6 - 2008-02-02
- Changed the example LoggingFilter into something actually usable. It now logs
just the raw messages (minus encryption)
from user sesssions (as RECV) and their responses from the proxy (as SENT).
- Removed all unnecessary java.util.logging experiments, to be compatible with
gcj/gij and possibly other jvms.
- Fixed: Removing au-users no longer requires restart.
- Fixed: Adding/removing/changing listen ports should now work without restart.
Disabling a profile will close the
ports (but existing sessions are not affected until kicked). Any change to a ca-
profile config will result in the
listen ports for that profile being closed and reopened.
- Fixed: Status-web httpd no longer logs to sysout if its log file is disabled
(removed from config).
- Fixed: Status-web jscript continues to request xml after connection errors (now
logs out instead).
- Fixed: Cache timestamp bug that could sporadically delete all cached ecms except
one.
- Fixed: ECM interval for radegast sessions (0 was always shown).
- Fixed: Mgcamd OSD message sending would fail if radegast sessions were active.
- Exposed the automatic rotation features of the logging api, in case anyone is
serious about using the logs for stats.
A maximum size for all file logs as well as a number of files to cycle through
can be specified (see below).
Additionally, any active file logs are now re-initialized when the config is
updated/touched.
- Added JVM stats to the status-web title for all views (os, version, heap
used/allocated, thread count etc).
- Enhanced the embedded httpd (keep-alive connections, gzip content-encoding).
- Changes to proxy.xml:
Changed: Renamed <exclude-services> setting to <reset-services>, to reflect what
it actually does.
Services in this list will also no longer cause "lost service" warnings.
Changed: <auto-exclude-threshold> is now <auto-reset-threshold>.
Added: Attribute 'bind-ip' to listen-ports. Optional local ip to bind listeners
to (default is all, i.e 0.0.0.0).
Added: Element <bind-ip> to <status-web>. Optional local ip to bind httpd listen
port to.
Added: Element <bind-ip> to <rmi>. Optional local ip to bind all rmi-related
listen ports to.
Added: Attributes 'rotate-count' and 'rotate-max-size' (in kb) to <log-file> (for
both main log and status-web).
E.g: setting count to 3 and max-size to 2048 will cycle between file.log.0,
file.log.1 and file.log.2 when they
reach 2 megs. The file currently in use will be indicated by a separate .lck
file.
Restart is required to change the log rotation.
0.7.5 - 2007-10-17
- Automatic profile assignment based on ca-id is now done for each cws reconnect,
not just the first one.
- No longer ignoring HD services when parsing (dvb service types 0x11 and 0x17, for
mpeg2 and "advanced codec" hdtv).
- "Successful" cannot decode replies are no longer counted as ecm failures in the
stats.
- Additionally tweaked load-sharing (0.7.3 may have had a serious bug that only
manifested under heavy load on 3+ cards).
- Emm's are now properly acknowledged to the client even when sent by non au-users
(without forward of course).
- Fixed several potential problems with radegast sessions (clients using radegast
towards the proxy).
- Added a delay on startup, listen ports will now not be opened until the cws
connector manager has had a chance to run
through one connection attempt for each active cws connector (this will prevent
the service mapper from removing
entries for "unknown" connectors that simply haven't been connected for the first
time yet).
- Added a real time negotiation procedure in the clustered cache to maximize use of
cache-sharing. Using this adds a
fixed delay (e.g 100 ms) to _every_ transaction. The time is used to collect
cache notifications for a pending ecm
from all proxies in the cluster, and ensure that only one of them proceeds with
querying a card. Highly experimental,
not quite sure yet whether this is useful in a real life scenario. The multiple
proxies will attempt to find which
one is best suited to handle a given request (based on estimated queue time and
whether the services is known to
exist on a local card or not). If in doubt, leave it alone (it's enabled with
<sync-period> for the clustered cache).
- Added a preconfigured java-service-wrapper setup for running the proxy as a
service on w2k/2k3/xp/vista. See README.
- The 'last-transaction' time per session now includes any time it took to send the
reply back to the client. This
means it is no longer depending exclusively on the proxy response time, but also
on the client connection.
- Changed CWS average processing time to current processing time (thats what it was
before as well, just a misleading
label). Average processing time is now the true total average for all ecm's
processed since connect (and utilization
will now show both versions).
- Made it possible to use any number of listen ports for each profile, each can
have their own protocol and allow/deny
lists as well as their own des-key/noencryption settings (and any other protocol-
specific data). Adding/changing ports
while running should work now too.
- Fixed weird synchronization issues with the session manager (caused NPE in
various places and blocked logins,
especially in conjuction with heavy use of the status web).
0.7.3 - 2007-07-28
- Fixed a bug that caused web logging to switch to sysout after changing the httpd
port number.
- Fixed problem with rapid cws reconnect that caused an IndexOutOfBoundsException
if there was pending traffic.
- Fixed NPE on NewcamdCws connect if the socket was unexpectedly closed during the
login procedure.
- Fixed last-transaction time for a user session getting confused by other message
types (non-ECM).
- Improved load balancing: queue size has been redefined to take the average
processing time into account, meaning that
in theory a queue size of 3 on a fast card can now be considered faster than a
queue size of 1 on a slow card.
- Updated the example web page to include several user contributions, including a
user/session section and re-use of
the xml parser object (to stop IE from leaking memory with each xml pull). I'm
keeping my logo though. :)
- Added a filename translation servlet for picon images. See
/picon/readme.picon.txt in the war for details.
0.7.2 - 2007-06-10
- Fixed a bug that would cause emm's to be forwarded to connectors that weren't
connected (caused NPE stacktrace in log).
- Fixed NPE in radegast response parsing when no matching pending request was
found.
- Fixed a serious cache sharing bug that could prevent remote cache data from being
used between servers with different
system locales set for the jvm (language/regional settings).
- Fixed another serious cache sharing bug that meant system clocks had to be
synchronized to within the max-cw-age time
across all servers, or cached cw's could be deleted before they were used.
- Added a cache-only (card-less) mode. A profile that makes use of this will accept
connections even if there are no
cws connectors ready (or even defined). It will rely entirely on cache sharing to
handle requests.
This allows a frontend proxy to be set up where untrusted users could get access
to anything cached, without being
able to affect the traffic load on any cards. Such a card-less frontend proxy
would only receive cache data (one way,
use the receive-only ClusteredCache setup by leaving out remote-host and remote-
port) from one or more other proxies,
thus completely isolating clients from the real proxies and their
clients/servers.
As long as the backend proxies have enough users to statistically ensure that all
the profiles services are cached
at any given moment, all services would also work for clients in the frontend
proxy.
NOTE: When there are no newcamd connectors in a profile, clients will receive
dummy card data on connect (empty).
This may not be good enough for all clients/ca-systems, but it works for me. If
you do have connectors defined
and cache-only mode set, the connectors will not be used for traffic (only for
card-data).
- Changes to proxy.xml:
Added: Attribute 'log-zapping' for <logging>. Set to false to disable the log
entries for when users switch service.
Added: Attribute 'cache-only' for <profile>. Set to true and the profile will
accept connections even without
card connectors. This will also prevent the "no available card" warnings, the
proxy will instead silently return
cannot decode for each cache miss.
Added: Attribute 'no-encryption' for <newcamd> (in profile). Setting this to true
means the proxy will handle
unencrypted newcamd traffic on this listen port. NOTE: no clients support this,
its just for debugging.
0.7.1 - 2007-05-19
- Added an option for completely disabling the service-mapping. Doing so means the
proxy will no longer attempt to find
out which services exist on each card. Instead it will assume all cards in the
profile are identical and only apply
load balancing. This way effective clusters can be achieved even with protocols
or clients that don't include SID,
providing the cards all have the same services. It can also be useful for
troubleshooting in small clusters.
Profiles for which the mapping has been switched off will not show any service
lists in the http/xml api or webgui.
0.6.3 - 2007-03-03
- Changes to proxy.xml:
Changed: Attribute 'provider' for <services-file> can now contain a list of
names (separated by space). Use this
in case services in the file have variations in the provider name (or
lists some as unknown even though
they should be part of the same provider subscription).
Added: <unknown-sid> to section mapper. Defines a special SID that will be sent
to servers when SID is 0 (unknown).
Client requests for this SID will also be treated as if it was 0. This
can be used as a workaround for
servers that require a non-zero SID, and for clients that send a fixed
special SID instead of 0 when service
is unknown (e.g cardlink). Make sure that this isn't set to a real SID
that exists in the services file.
Can be specified globally or per profile like all other mapper elements.
0.6.2 - 2007-02-10
0.6.1 - 2007-02-05
- Added a view of the 'watched-services' to the channels section of the example web
page.
- Added cache-status to the status section of the example web page.
- Added some reasonable usage stat counters to DefaultCache and ClusteredCache.
- Added isMapExcluded() to UserManager interface, return true to stop a user from
discovering new services or changing
the status of existing ones (map failure counters etc will not be updated as a
result of ecms from this user).
- Improved some of the INFO level logging to more clearly show the new load
balancing and timeout handling in action.
- UserManager interface now has limits that can be imposed on idividual users, but
SimpleUserManager will _not_ use this.
For anyone working on their own user manager, the methods are:
Set getAllowedServices(String name, String profile); // return Set of Integer
(service ids, null for all)
Set getAllowedConnectors(String name); // return Set of String (connector
names, null for all)
int getAllowedEcmRate(String user); // return minimum interval between ecm in
seconds (-1 for no limit)
A note about allowed connectors: this will stop ecm's from this user from being
routed to other connectors, but it
will _not_ stop them from watching services that only exist on those excluded
connectors, through the cache.
- Proxy now detects the type of client for each connected user session.
- Fixed xml-related problems with http/xml api and java 1.5.
- "Webgui" cleanup, removed everything not used or not working.
- Fixes to startup sequence, all errors shown by the start script should now halt
the proxy with a non-zero exit code.
- Fixed enigma services file parsing (0xffffff transponder ids and other values
caused it to fail).